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EXECUTIVE  SUMMARY 


On  28  -  30  July  2008,  a  workshop  was  conducted  on  the  campus  of  the  National  Defense  University  (NDU)  to 
characterize  the  capabilities  needed  to  perform  effective  Human,  Social,  Culture  Behavioral  (HSCB)  modeling  in 
support  of  operational  users  and  senior  decision  makers.  The  workshop  was  sponsored  by  Dr.  Robert  Foster, 
Director,  BioSystems,  in  the  Office  of  the  Director,  Defense  Research  &  Development  (DDR&E).  The  workshop 
was  organized  and  conducted  by  the  Center  for  Technology  and  National  Security  Policy  (CTNSP),  NDU.  To 
achieve  the  primary  goal  of  the  workshop,  approximately  120  participants  were  assembled  from  the  social 
sciences  and  operations  analyses  communities.  Participants  were  drawn  from  government,  academia,  industry, 
Federally  Funded  Research  &  Development  Centers  (FFRDCs)  and  University  Affiliated  Research  Centers. 

The  workshop  was  divided  into  three  parts.  The  first  part  consisted  of  plenary  briefings,  which  characterized  the 
nature  of  the  problem,  depicted  the  state-of-the-practice,  and  identified  the  steps  needed  to  achieve  desired 
capabilities.  The  second  and  third  parts  were  organized  as  working  groups.  On  the  first  day  of  the  workshop,  the 
participants  were  assigned  to  discipline  panels  (i.e.,  Social  Sciences  (Micro);  Social  Sciences  (Macro);  Operations 
Research  (methodologies  and  tools);  Operations  Research  (data  and  verification,  validation,  and  accreditation 
(VV&A)).  On  the  second  day  of  the  workshop,  the  participants  were  assigned  to  problem-domain  panels  to 
address  issues  posed  by  operational  users  and  senior  decision  makers  (e.g.,  deterrence;  counterterrorism;  counter 
insurgency;  stability,  security,  transition,  reconstruction  (SSTR)  operations).  In  each  case,  the  groups  were  asked 
to  characterize  the  capabilities  needed  to  satisfy  the  interests  of  operational  users  and  senior  decision  makers. 

This  Executive  Summary  synthesizes  the  insights  from  these  three  perspectives  to  develop  a  holistic  picture  of 
needed  capabilities. 

Key  Needs 

Ultimately,  HSCB  modeling  needs  were  classified  into  twelve  interrelated  categories.  As  noted  in  the  figure 
below,  the  driver  for  the  needs  arises  from  the  representative  questions  from  operational  users  and  senior  decision 
makers.  There  are  four  key  categories  where  the  needs  transcend  disciplinary  needs:  data,  VV&A,  education  and 
training,  and  outreach.  Four  categories  are  of  particular  concern  to  the  social  sciences:  definitions,  basic  research, 
Measures  of  Merit  (MoMs),  and  theories.  Four  categories  are  of  particular  concern  to  operations  researchers: 
tools,  representation  in  tools,  “predictions”  or  forecasts,  and  design  of  experiments.  Each  category  is  summarized 
below. 


Social  Sciences 


Capabilities  Needed  for  HSCB  Modeling 


Questions.  Many  of  the  questions  from  operational  users  and  senior  decision  makers  can  be  assigned  to  five 
areas.  First,  there  is  interest  in  predicting  a  future  state.  For  example,  there  may  be  interest  in  predicting  refugee 
flows,  contagion  of  disease,  or  authenticity  of  cultural  change.  Second,  there  is  interest  in  supporting  an  activity. 
As  an  example,  analysts  are  frequently  asked  to  support  the  generation  of  Theater  Security  Cooperation  Plans 
(TSCPs).  Third,  there  is  often  the  need  to  balance  competing  activities.  For  example,  a  request  may  be  made  to 
balance  flow  through  the  SSTR  process  (e.g.,  characterize  which  activity  should  be  done,  when).  Fourth,  there  is 
interest  in  prioritizing  among  competing  options.  This  can  range  between  investing  resources  (e.g.,  among  S&T 
options)  to  support  to  operations  (e.g.,  which  SSTR  operations  to  undertake).  Finally,  there  is  pervasive  interest  to 
understand  the  context  or  root  causes.  For  example,  there  is  interest  in  understanding,  inter  alia,  failed  states, 
legitimacy,  why  people  become  insurgents,  deterrence/influence  /containment,  unintended  consequences,  and 
governance.  Consistent  with  these  questions,  the  participants  at  the  workshop  identified  the  following  HSCB 
modeling  needs. 

Data  Needs.  The  workshop  participants  identified  six  key  themes  for  data  needs.  First,  there  is  a  need  to  develop 
appropriate  HSCB  taxonomies  and  ontologies.  Second,  it  was  observed  that  there  is  a  need  to  implement  efforts  to 
tailor  HSCB  data  to  satisfy  the  intended  purposes.  Third,  it  is  important  to  perform  and  record  data  V&V  efforts 
(e.g.,  integrity,  consistency,  reliability,  source)  as  metadata.  Fourth,  it  is  vital  to  avoid  “stale”  data.  Thus,  there  is  a 
need  to  update  local  and  national  data,  with  appropriate  periodicity.  Fifth,  the  complexity  of  the  questions  requires 
that  we  capture  data  capabilities  in  many  dimensions.  As  a  partial  listing,  we  need  data  in  the  areas  of  the 
environment,  medical,  attitudes,  affiliations,  and  legal  systems.  Finally,  given  the  dispersion  of  the  data,  it  was 
recommended  that  we  perform  an  assessment  of  the  desirability  of  a  Central  HSCB  Data  Repository.  That  study 
should  address  a  variety  of  issues  including  classification,  access,  open  source  data,  legal,  granularity,  qualitative 
data,  maintenance,  and  dissemination. 

Definition  Needs.  At  the  workshop,  it  was  noted  that  for  HSCB  modeling  the  social  scientists  needed  to  develop 
more  specific  definitions  and  define  commonalities  across  disciplines  to  drive  action.  Specifically,  there  is  a  need 
to  develop  a  variety  of  products  including,  inter  alia,  a  library  of  ontologies,  semantic  descriptions,  a  thesaurus,  a 
dictionary,  data  lexicons  with  metadata,  and  standards.  Many  of  these  needs  overlap  with  the  data  needs,  cited 
above. 

Measures  of  Merit  (MoMs)  Needs.  Historically,  the  operations  research  community  has  been  comfortable  with 
the  concept  of  formulating  MoMs  subsuming  Measures  of  Performance  (MoPs)  and  Measures  of  Effectiveness 
(MoEs).  However,  practitioners  of  the  social  sciences  are  less  familiar  with  this  approach.  Thus,  we  need  to  tailor 
the  MoMs  to  HSCB  problems  of  interest  and  develop  relationships  that  link  key  MoPs  and  MoEs.  Furthermore, 
we  need  to  display  HSCB  MoMs  to  operational  users  and  senior  decision  makers  in  a  fashion  that  conveys 
appropriate  levels  of  uncertainty  and  risk. 

Theory  Needs.  As  demonstrated  at  the  workshop,  the  social  sciences  have  formulated  competing  theories  for 
many  subjects  of  interest  (e.g.,  root  causes  of  terrorism).  When  multiple  theories  exist,  we  need  codes  of  best 
practice/guidelines  on  which  theory  to  use,  when.  In  addition,  there  are  many  areas  where  the  social  sciences  have 
not  yet  developed  theories  in  forms  useful  for  HSCB  modeling.  We  need  to  develop  appropriate  social  sciences 
theories  to  address  these  key  gaps  or  mismatches. 

Basic  Research  Needs.  There  are  many  areas  of  interest  to  national  security  where  basic  research  in  the  social 
sciences  must  be  performed  to  support  HSCB  modeling.  For  example,  we  need  studies  of  influencers  on 
attitude/behavior  of  civilians  based  on  ethnic,  tribal,  cultural,  religious,  and  political  considerations.  It  is  important 
to  develop  a  comprehensive  list  of  these  areas  where  basic  social  science  research  is  needed  and  to  prioritize  this 
list  from  a  national  security  perspective. 

Tools  Needs.  There  was  broad  agreement  at  the  workshop  that  we  require  an  expanded  set  of  HSCB  tools. 
However,  the  proper  architecture/framework  for  these  HSCB  tools  is  an  open  question.  The  workshop  did, 
however,  elicit  the  following  needs  with  respect  to  tools:  the  suite  of  tools  should  include  a  balanced  mix  of 
techniques  that  take  advantage  of  the  strengths  of  the  tools  while  ameliorating  their  weaknesses  (e.g.,  system 
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dynamic  models;  agent  based  models;  wargames);  consideration  should  be  given  to  creating  a  “collaborative 
environment”  in  which  selected  models  can  be  linked/federated  and  evaluated  (particularly  with  respect  to 
“precision”);  consider  the  use  of  a  “service  bus”  or  Global  Information  Grid  for  data  repositories;  ensure  that 
models  are  tailorable;  employ  hierarchical  modeling  with  meta-model/meta-data  aggregation/disaggregation;  and 
assemble  a  resource  repository  of  models  and  data. 

Representations  in  Tools  Needs.  The  question  in  this  area  is  what  real-world  factors  should  the  tools  represent? 
The  participants  at  the  workshop  warned  against  “mirror-imaging”.  Thus,  it  is  important  that  we  use  creativity  in 
modeling  ourselves  as  well  as  “others”.  In  addition,  the  workshop  subdivided  the  social  scientists  into  the 
categories  of  “micro”  and  “macro”  representations.  This  is  an  artificial  distinction  and  we  need  to  provide 
feedback  between  “micro”  and  “macro”  representations.  Overall,  there  is  a  need  to  capture  phenomena  from 
multiple  perspectives.  These  include  organizational  performance,  cultures  and  institutions,  all  types  of  operations, 
and  situational  awareness  of  all  parties. 

Prediction  Needs.  Above,  we  cited  representative  questions  that  might  be  posed  by  the  operational  users  and 
senior  decision  makers.  It  is  important  to  clarify  the  extent  to  which  we  can  perform  HSCB  modeling  to  “predict” 
outcomes.  As  an  example,  at  least  four  possible  levels  of  prediction  are  envisioned.  These  include:  “hard” 
predictions  of  events  (with  “error  bars”  to  characterize  uncertainty);  “soft”  predictions  of  likelihoods  (e.g.,  for 
multiple  possible  results);  explorations  of  possibilities  (e.g.,  “what  if...?”);  or  situational  awareness  and 
understanding.  For  this  reason,  “forecasting”  is  a  better  term  to  use  than  “predicting.” 

Design  of  Experiments  Needs.  It  was  clear  from  the  workshop  presentations  that  many  of  the  users  of  HSCB 
models  were  not  familiar  with  efficient,  effective  designs  of  experiments.  We  should  draw  on  the  insights 
developed  in  academia  (e.g.,  at  NPS  for  M&S  characterized  by  large  numbers  of  variables)  to  characterize  the 
interesting  parts  of  response  surfaces  using  innovative  experimental  designs  (e.g.,  focused  fractional  factorial 
designs). 

Verification,  Validation  &  Accreditation  (VV&A)  Needs.  One  of  the  key  themes  of  the  workshop  was  the  need 
to  perform  VV&A  for  HSCB  models.  To  achieve  that  objective,  there  is  a  need  to  generate  guidelines  that  enable 
us  to  perform  V&V  credibly,  with  acceptable  levels  of  resources.  Historically,  the  operations  research  community 
has  focused  on  V&V  of  models  and  data.  However,  the  workshop  emphasized  that  the  social  sciences  pose 
additional  challenges  in  order  to  V&V  relevant  theories  and  Subject  Matter  Experts  (SMEs).  Note  that  the  V&V 
process  must  be  documented,  transparently,  to  facilitate  implementation  of  the  accreditation  function.  In  addition, 
the  working  group  on  VV&A  recommended  that  we  use  V&V  to  create  “tags”  for  theories,  methods,  models,  and 
data  to  allow  retrieval  of  desired  item  when  needed.  Furthermore,  there  is  a  need  to  create  open  data  on  the 
detailed  VV&A  status  of  models  and  data. 

Education  and  Training  (E&T)  Needs.  Three  E&T  needs  emerged  from  the  workshop.  First,  there  is  a  need  to 
augment  the  curriculum  for  social  scientists  and  operations  analysts  to  give  them  adequate  education  to  enhance 
cross-discipline  communication.  Second,  it  is  vital  to  create  and  sustain  a  HSCB  Community  of  Interest  (COI)  to 
foster  high  performance,  multidisciplinary  teams.  This  COI  should  provide  continuity  of  action  using  a  variety  of 
approaches  (e.g.,  electronic  interaction;  face-to-face  meetings).  This  continuity  might  be  achieved  by  drawing  on 
MORS’  proposed  Social  Science  COI.  Lastly,  there  is  a  need  to  develop  the  tools  and  data  needed  to  “train  as  we 
fight”  to  support  both  the  E&T  and  operational  communities. 

Outreach  Needs.  The  workshop  highlighted  three  areas  where  enhanced  HSCB  modeling  outreach  is  needed. 
First,  it  is  vital  to  expand  the  HSCB  COI  to  include  balanced  interagency  participants  (e.g.,  National  Security 
Council,  US  Institute  of  Peace,  Department  of  State,  Department  of  Justice)  Second,  there  is  a  need  to  participate 
actively  in  international  forums  that  address  HSCB  modeling  issues.  These  would  include  NATO  SAS  initiatives 
on  HSCB  and  Irregular  Warfare.  Finally,  there  is  great  sensitivity  to  the  attitudes  of  many  members  of  the  social 
sciences  community  who  believe  that  it  would  be  unethical  to  work  with  the  DoD.  To  address  that  issue,  it  is 
important  to  develop  a  compelling  narrative  to  explain  the  value  of  a  collaborative  relationship  between  the  social 
sciences  community  and  DoD. 


v 


Where  Do  We  Go  From  Here? 


This  workshop  was  extremely  successful  in  generating  baseline  needs  for  the  HSCB  modeling  community.  It  will 
be  followed  by  two  subsequent  workshops  that  will  build  on  its  achievements.  In  Workshop  2,  the  participants 
will  be  asked  to  characterize  the  state-of-the-art  in  HSCB  modeling  with  respect  to  the  twelve  categories  cited 
above.  They  will  also  be  asked  to  compare  the  state-of-the-art  to  these  needs  to  identify  major  gaps.  Although  the 
precise  criteria  have  yet  to  be  finalized,  there  is  interest  in  assessing  the  risks  associated  with  these  gaps  (e.g., 
their  severity  and  frequency). 

In  Workshop  3,  the  participants  will  be  asked  to  assess  the  resources  that  are  needed  to  mitigate  those  gaps.  Based 
on  the  risk  assessments  and  resource  needs,  it  should  be  feasible  to  identify  “low  hanging  fruit”  (e.g.,  significant 
gaps  for  which  limited  resources  are  needed)  as  well  as  vital  gaps  (e.g.,  gaps  that  are  catastrophic  or  critical  and 
occur  frequently).  The  results  of  those  deliberations  will  be  used  to  generate  an  initial  version  of  a  Science  & 
Technology  Roadmap  for  HSCB  modeling.  It  is  anticipated  that  this  roadmap  will  be  a  “living  document”  that 
evolves  as  we  gain  a  deeper  understanding  of  the  problem. 

Subsequently,  the  results  of  the  workshops  will  be  used  to  justify  the  allocation  of  resources  for  HSCB  modeling 
to  OSD  and  Congress.  As  necessary,  follow-on  workshops  will  be  conducted  to  characterize  progress  and  to  re¬ 
evaluate  priorities. 

—  Dr.  Stuart  Starr,  Synthesis  Group  Chair 
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1.  BACKGROUND 


The  Director,  BioSystems,  DDR&E,  is  sponsoring  work  on  Human,  Social,  and  Cultural  Behavior  (HSCB).  As 
part  of  this  work,  the  Center  for  Technology  and  National  Defense  Policy  (CTNSP)  of  the  National  Defense 
University  (NDU)  is  organizing  and  conducting  three  workshops.  These  workshops  investigate  the  needs  for 
HSCB  modeling,  the  current  state  of  the  art  in  HSCB  modeling  and  the  gaps  between  the  needs  and  current 
modeling,  and  formulate  a  Science  and  Technology  (S&T)  roadmap  to  address  the  highest  priority  gaps.  The 
results  of  the  workshops  will  be  used  to  justify  the  allocation  of  resources  for  HSCB  modeling  to  the  Office  of  the 
Secretary  of  Defense  (OSD)  and  the  Congress.  This  report  describes  the  results  of  the  first  workshop,  “Needs.” 

The  needs  workshop  was  held  on  June  28  -  30,  2008  at  NDU.  Approximately  120  people  attended  the  by¬ 
invitation-only  workshop.  The  participants  were  selected  from  the  government,  academia,  industry,  and  the 
Federally  Funded  Research  and  Development  Centers  (FFRDCs).  The  stated  purpose  of  the  workshop  was  the 
following: 

Characterize  the  capabilities  needed  to  perform  effective  HSCB  modeling  in  support  of 
operational  users  and  senior  decision  makers. 

A  secondary  purpose  was  the  opportunistic  capture  of  additional  insights  of  the  participants  that  would  be  helpful 
in  supporting  HSCB  modeling. 

1.1  WORKSHOP  ORGANIZATION 

Michael  Baranick,  Senior  Research  Fellow  at  NDU,  organized  and  chaired  the  workshop.  He  organized  it  in  three 
phases:  pre-workshop,  workshop,  and  post-workshop. 

The  pre-workshop  phase  included  the  production  of  papers  and  presentations  by  plenary  speakers  and  working 
group  chairs  and  co-chairs  to  be  distributed  prior  to  the  workshop  to  attendees.  These  papers  were  designed  to  set 
the  stage  for  the  workshop  and  create  a  common  understanding  among  the  participants,  increasing  the  amount  of 
time  available  for  productive  work  in  the  workshop  phase. 

The  workshop  was  organized  with  plenary  sessions,  working  group  breakout  sessions,  and  a  wrap-up  session. 
The  first  two  days  consisted  of  plenary  sessions  in  the  morning  and  working  group  sessions  in  the  afternoon  and 
involved  all  of  the  participants.  The  final  half-day  session  was  attended  only  by  the  working  group  chairs  and  co¬ 
chairs  and  the  Synthesis  Group. 

The  plenary  sessions  consisted  of  presentations  by  leaders  in  various  fields.  The  purposes  of  the  presentations 
were  to  characterize  the  nature  of  the  HSCB  modeling  problem,  depict  the  state-of-the-practice  of  HSCB 
modeling,  and  describe  the  steps  needed  to  achieve  the  desired  capabilities. 

The  working  groups  were  constructed  with  two  divisions.  The  first  day’s  working  groups  were  divided  by 
discipline:  micro-social  sciences,  macro-social  sciences,  operations  research  (OR),  and  Verification,  Validation 
and  Accreditation  (VV&A)  of  models,  data,  methods,  and  theories.  The  second  day’s  working  groups  were 
divided  by  problem  set:  deterrence,  counterterrorism  (CT),  counter  insurgency  (COIN),  and  Stability,  Security, 
Transition,  and  Reconstruction  Operations  (SSTRO).  The  working  groups  were  not  segregated  by  individual 
specialties,  but  were  composed  of  mixes  of  specialties  to  ensure  full  communication  and  surfacing  of  ideas  from 
all  points  of  view.  In  addition,  each  working  group  had  participation  by  members  of  the  Synthesis  Group  to 
ensure  that  valuable  connections  could  be  made  among  the  separate  sets  of  results.  The  wrap-up  session  was 
designed  to  permit  the  chairs  and  co-chairs  of  the  working  groups  to  gather  and  digest  the  results  from  their 
individual  sessions  and  present  them  to  each  other. 
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Figure  1.  Dividing  the  HSCB  Modeling  Problem 


The  post-workshop  phase  included  two  parts.  The  first  part  was  the  polishing  of  the  individual  papers  and 
working  group  reports,  including  the  Synthesis  Group  report,  with  its  overall  view  of  the  results  of  the  workshop. 
This  first  part  also  included  the  production  of  the  Final  Report  Briefing,  this  Final  Report  document,  and  the 
creation  of  a  book  detailing  the  results  of  the  workshop.  The  second  part  of  the  post-workshop  phase  consisted  of 
the  production  of  ideas  for  FISCB  modeling  tasks,  both  from  the  social  sciences  and  the  modeling  points  of  view. 

1.2  DETAILS  OF  THE  WORKSHOP 

The  attendees  were  a  diverse  group,  as  shown  in  the  list  below.  The  original  expectation  was  that  about  80  of 
those  invited  would  be  able  to  attend.  The  50%  larger  than  expected  attendance  indicated  that  they  were  also 
motivated. 


•  Government 

o  OSD/Joint  Staff  (e.g.,  DDR&E,  OASD(PA&E),  OUSD(Policy)) 
o  COCOMs/Services  (e.g.,  JFCOM,  NRL,  AFRL) 
o  Academia  (e.g.,  NDU,  USMA,  AFIT,  NPS,  AWC) 
o  Interagency  (e.g.,  USIP,  DOS,  DHS) 

•  FFRDCs/UARCs  (e.g.,  IDA,  MITRE,  RAND,  JHU/APL) 

•  Industry  (e.g.,  Sentia,  IBM,  BAH,  Phase  One  Communications,  Group-W,  SAIC,  EBR,  Hartley 
Consulting) 

•  Academia  (e.g.,  University  of  Maryland,  University  of  Pennsylvania,  GMU,  Florida  State  University, 
Purdue,  University  of  California  (San  Diego),  GWU,  ODU,  Virginia  Tech) 

•  Non-profit  (e.g.,  Potomac  Institute,  Alaska  Native  Heritage  Center) 

•  International  (e.g.,  Canada) 
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The  plenary  presentations  were  meant  to  ensure  that  the  working  groups  had  relevant,  fresh  material  to  stir  their 
thinking  processes.  Clearly  they  succeeded,  as  the  large  number  of  questions  from  the  audience  showed.  The 
plenary  speakers  and  presentation  titles  are  shown  below. 


•  Context  presentations 

o  Bob  Foster,  DDR&E 
o  Andre  van  Tilborg,  DUSD,  S&T 

•  Patricia  Partnow,  Dean  Hartley,  “Using  Cultural  Information  to  Model  DIME/PMESII  Effects” 

•  David  Garment,  “Approaches  to  Country  Risk  Analysis  and  Early  Warning” 

•  Eli  Berman,  “Sects  and  Violence  for  Economists” 

•  Will  Moore,  “Cross-National  Correlates  of  Terror:  An  Economical  Inquiry  of  the  Late  20th  Century” 

•  David  Siegel,  “Social  Network  Structures  and  Counterinsurgency  /  Counterterrorism. . .” 

•  Barry  Silverman,  “Systems  Engineering  is  the  New  Social  Science” 

•  Thomas  Ferleman,  “Modeling  Global  Futures. . .” 


The  leaders  of  the  working  groups  were  chosen  from  among  the  experts  of  the  respective  fields.  Figure  2  shows 
the  disciplinary  working  groups  and  their  leadership.  Figure  3  shows  the  problem  domain  working  groups  and 
their  leadership. 


Disciplines 

Chair 

Co-Chair 

Social  Science  -- 
Micro 

Jerry  Post 

Larry  Kuznar 

Social  Science  -- 
Macro 

Richard  Lobban 

Lauren  Cobb 

OR  (Theories) 

Mike  McGinnis 

Bob  Sheldon 

OR  (W&A) 

Dean  Hartley 

Marina  Arbetman- 
Rabinowitz 

Figure  2.  Disciplinary  Working  Groups  and  Leaders 
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Problem 

Chair 

Co-Chair 

Deterrence 

Pat  McKenna 

Yuna  Wong 

COIN 

Paul  Goldstone 

CDR  Brett  Pierson 

CT 

LTC  Lailari 

Dr.  Gupta 

SSTRO 

Paul  West 

Dick  Deckro 

Figure  3.  Problem  Domain  Working  Groups  and  Leaders 


The  Synthesis  Group  was  recruited  from  those  knowledgeable  in  HSCB  theory  and  modeling  and  from  those  with 
experience  in  the  Military  Operations  Research  Society’s  (MORS)  synthesis  group  procedures.  Several  of  its 
members  met  both  criteria.  The  members  are  shown  below. 


•  Stuart  Starr,  NDU  (Chair) 

•  Dean  Hartley,  Hartley  Consulting  (Co-Chair) 

•  Myriam  Abramson,  NRL 

•  Chuck  Barry,  NDU 

•  Doug  Clark,  NRL 

•  Skip  Cole,  USIP 

•  Dave  Davis,  GMU 

•  Margaret  Hayes,  EBR 

•  Richard  Hayes,  EBR 

•  Roger  Hillson,  NRL 

•  Pat  McKenna,  STRATCOM 

•  A1  Sciarretta,  CNSTI 

•  Ted  Woodcock,  GMU 
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2.  DETAILED  RESULTS 


The  results  of  the  workshop  are  divided  into  three  sections,  Plenary  Presentations,  Disciplinary  Working  Group 
results  and  Problem  Domain  Working  Group  results. 

2.1  PLENARY  PRESENTATIONS 

There  were  nine  plenary  presentations.  Each  presentation  is  briefly  described  in  this  section. 

2.1.1  Foster 

Dr.  Bob  Foster,  the  sponsor  of  these  workshops  and  Director,  BioSystems,  DDR&E  opened  the  workshop  with 
the  statement,  “The  social  sciences  are  the  HARDER  sciences.”  His  principal  point  was  that  in  order  to  deal  with 
the  issue  of  HSCB  modeling,  it  will  be  necessary  to  develop  a  multi-disciplinary  approach  that  involves  multiple 
partnerships,  both  domestically  among  academia,  government,  and  industry  and  internationally  between  the  US 
and  its  allies,  such  as  NATO  Allied  Command  Transformation  (ACT). 

2.1.2  Van  Tilborg 

Dr.  Andre  van  Tilborg,  the  Deputy  Under  Secretary  of  Defense  (Science  and  Technology)  (DUSD(S&T)),  serves 
as  the  principal  advisor  and  assistant  to  the  Director  of  Defense  Research  and  Engineering  (DDR&E)  for  all 
scientific  and  technical  matters.  Dr.  van  Tilborg  gave  the  keynote  address. 

He  stated  that  the  key  challenges  in  performing  HSCB  modeling  include  acquiring  evidence-based,  convincing 
data  (vice  opinions)  and  developing  a  suite  of  tools  that  can  be  transitioned  to  the  warfighter.  With  respect  to  the 
data,  the  main  problems  are  in  knowing  where  to  look  for  the  data  and  getting  access  to  it.  However,  there  is 
another  relevant  challenge,  that  is  to  convince  OSD  and  Congress  that  this  endeavor  is  worth  doing.  Van  Tilborg 
suggested  that  the  principal  argument  was  stated  by  Sun  Tzu,  “know  your  enemy!” 

2.1.3  Partnow  and  Hartley 

Dr.  Patricia  Partnow  is  an  anthropologist  and  is  the  Vice  President  of  Cultural  and  Educational  Programs  at  the 
Alaska  Native  Heritage  Center  in  Anchorage,  Alaska.  Dr.  Dean  S.  Hartley  III  is  the  Principal  of  Hartley 
Consulting  and  has  done  work  in  HSCB  modeling  for  a  dozen  years.  Partnow  and  Hartley  presented  the  paper, 
“Using  Cultural  Information  to  Model  DIME/PMESII  Effects.” 

Creating  models  that  are  based  on  cultural  information  is  difficult,  but  not  impossible.  Partnow  and  Hartley 
proposed  a  methodology  that  melds  the  strengths  of  anthropology  and  operations  research  to  provide  a  holistic 
approach  to  DIME/PMESII  (diplomatic,  information,  military,  economic/political,  military,  economic,  social, 
information,  infrastructure)  modeling.  The  anthropologist  starts  with  the  people  and  looks  for  those  areas  that 
form  culturally  important  categories  for  the  people  being  studied,  such  as  cosmology,  leadership,  land  use,  social 
control,  and  affiliation.  Deep  ethnographies  are  used  to  create  an  anthropological  model.  A  top-down  model  of 
the  DIME/PMESII  system  is  then  connected  to  the  relevant  anthropological  models  for  an  area. 

2.1.4  Carment 

Dr.  David  Carment  is  Professor  of  International  Affairs  at  the  Norman  Paterson  School  of  International  Affairs, 
Carleton  University,  and  principal  investigator  for  the  Country  Indicators  for  Foreign  Policy  project  (CIFP). 
Carment’ s  primary  research  interests  fall  into  the  categories  of  conflict  prevention,  conflict  resolution,  ethnic 
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conflict,  peacekeeping,  internet  technology,  and  Canadian  foreign  affairs.  He  presented  “Approaches  to  Country 
Risk  Analysis  and  Early  Warning.” 

His  paper  laid  out  a  framework  for  country  risk  analysis  and  early  warning  drawing  on  the  methodologies 
developed  by  CIFP.  Special  attention  was  given  to  the  identification  of  failed  and  fragile  states.  The  paper  was  in 
four  parts.  In  the  first  section,  he  provided  some  concepts  and  definitions  as  they  relate  to  early  warning,  state 
failure  and  country  risk.  He  analytically  distinguished  between  fragility  (e.g.,  development,  conflict,  stability) 
and  failure.  Carment  noted  that  conflict  is  a  symptom,  not  a  cause,  of  fragility  and  made  the  observation  that 
autocratic  states  are  not  the  most  fragile.  The  second  section  assessed  the  strengths  and  weakness  of  specific 
approaches  to  country  risk  analysis.  In  the  third  section  he  examined  structural  risk  indicators  and  events  based 
monitoring.  The  main  issue  is  what  criteria  should  be  used  to  rank-order  nations  with  respect  to  key  factors,  e.g., 
legitimacy,  authority,  and  capacity.  The  fourth  and  final  section  concluded  by  discussing  the  integrated  product. 
Carment  emphasized  the  need  to  get  policy  makers  involved;  in  order  to  be  relevant,  you  must  speak  their 
language! 

2.1.5  Berman 

Dr.  Eli  Berman  of  the  Department  of  Economics,  University  of  California,  San  Diego  and  the  National  Bureau  of 
Economic  Research.  Berman’s  research  interests  are  Labor  Economics,  Labor  Demand  and  Technological 
Change,  Religion,  Middle  East,  Terrorism  and  Insurgency,  Fertility,  Immigration,  and  Language.  He  presented 
the  results  from  two  studies. 

The  first,  “Religion,  Terrorism  and  Public  Goods:  Testing  the  Club  Model,”  addressed  the  question,  “Can  rational 
models,  once  theological  explanations  are  discredited,  explain  why  certain  radical  religious  rebels  are  so 
successful  in  perpetrating  suicide  attacks?”  The  fundamental  barrier  to  success  turns  out  not  to  be  recruiting 
suicide  attackers;  there  is  a  rational  basis  for  volunteering.  Rather,  the  barrier  is  the  danger  of  other  operatives 
defecting.  A  club  model,  portraying  voluntary  religious  organizations  as  efficient  providers  of  local  public  goods 
explains  how  they  weed  out  potential  defectors  by  requiring  sacrifices  as  signals  of  commitment.  They  are 
thereby  able  to  succeed  in  risky  terrorist  attacks.  The  model  has  testable  implications  for  tactic  choice  and 
damage  achieved  by  clubs  and  other  rebel  organizations.  Data  spanning  a  half-century  on  both  terrorists  and  civil 
war  insurgents,  much  from  Middle  East  sources  and  Israel/Palestine,  reveal  that:  a)  missions  organized  by  radical 
religious  clubs  that  provide  benign  local  public  goods  in  the  absence  of  competing  provision  by  government  are 
both  more  lethal  and  are  more  likely  to  be  suicide  attacks  than  missions  organized  by  other  terrorist  groups  with 
similar  aims  and  theologies;  and  b)  suicide  attacks  are  chosen  when  targets  are  "hard,"  i.e.,  difficult  to  destroy. 
The  results  suggest  benign  tactics  to  counter  radical  religious  terrorism  and  insurgency. 

The  “Hearts  and  Minds”  model  helped  define  the  effects  on  victims  of  terrorist  violence  and  the  relationship 
between  Commander's  Emergency  Response  Program  (CERP)  spending  and  reduction  in  violence. 

2.1.6  Moore 

Dr.  Will  H.  Moore  is  Professor  of  Political  Science,  Florida  State  University.  Moore's  area  of  expertise  is  the 
scientific  study  of  violent  political  conflict.  He  presented  the  paper,  “Cross-National  Correlates  of  Terror: 
Empirical  Analysis  of  the  Late  20th  Century.” 

Moore  stated  that  data  limitations  have  prevented  scholars  from  developing  a  strong  sense  of  the  cross-national 
covariates  of  terror.  A  handful  of  such  studies  exist  using  the  ITERATE  data  set,  but  those  data  exclude  all 
domestic  terror  events,  which  is  to  say  most  such  events.  Further,  those  studies  suffer  from  a  common  weakness 
in  cross-national  studies  of  violence:  they  focus  on  the  characteristics  of  polities,  economies  and  societies  while 
ignoring  the  behavior  of  governments  and  dissidents.  Moore’s  study  addressed  both  weaknesses  by  [1]  utilizing  a 
new  dataset  (the  Global  Terror  Database)  and  [2]  including  the  behavior  of  governments  and  dissidents  within  the 
context  of  the  national  characteristics  that  influence  their  behavior.  The  empirical  domain  is  all  countries  in  the 
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world  from  the  mid-1970s  through  the  1990s.  Moore  identified  the  variables  that  influence  the  amount  of 
dissident  terror.  He  also  emphasized  that  “terror  is  a  tactic.”  He  concluded  that  his  work  provided  support  for 
Paul  Wilkinson’s  arguments  (e.g.,  value  of  liberal  democracy). 

2.1.7  Siegel 

Dr.  David  A.  Siegel  is  an  Assistant  Professor  of  Political  Science,  Florida  State  University.  His  work  uses  formal 
and  computational  models  to  explore  the  organizational  dynamics  of  terrorist  organizations,  and  the  effect  of 
repression  and  the  media  on  participation  in  collective  actions  such  as  insurgency  and  terrorism.  Siegel  presented 
the  paper,  “Social  Network  Structure  and  Counterinsurgency/Counterterrorism:  Using  Theory  to  Limit  Causal 
Links  and  Aid  in  Strategic  Planning.” 

According  to  Siegel,  both  qualitative  and  quantitative  methods  have  utility  in  political  forecasting,  yet  they  are 
often  treated  independently.  Siegel  argued  for  an  approach  that  formalizes  the  insights  inherent  to  qualitative 
models,  and  so  provides  for  their  quantitative  measurement  and  their  broader  generalization.  The  modeling 
approach  he  offered  treats  individuals  as  heterogeneous  in  intent,  and  influenced  by  their  interactions  with  others 
in  their  social  networks.  One’s  location  within  these  networks  affects  one’s  individual  behavior;  the  large-scale 
structure  of  the  networks  influences  aggregate  behavior.  Siegel  offered  a  typology  of  qualitative  network  types  to 
classify  how  different  networks  affect  aggregate  outcomes,  and  incorporate  the  actions  of  an  external  repressor  to 
explore  the  way  in  which  participation  in  collective  actions  may  be  limited  by  outside  pressure.  After  analyzing 
the  role  of  both  violent  and  non-violent  repression,  he  illustrated  which  types  of  data  on  social  networks  are  most 
useful  to  forecasting  within  each  network  type.  Siegel  also  showed  that  the  form  of  repression  is  often  less 
important  in  understanding  particulars  than  details  of  social  connections  or  prior  motivations.  This  fact  allows  for 
application  to  substantive  cases  with  little  available  data.  The  presentation  concluded  with  an  application,  using 
information  known  prior  to  the  event  to  forecast  the  differential  effect  of  repression  on  turnout  in  the  2005  Iraqi 
Legislative  elections. 

2.1.8  Silverman 

Dr.  Barry  G.  Silverman  is  Professor  of  Electrical  and  Systems  Engineering  at  the  University  of  Pennsylvania 
where  he  is  also  Director  of  the  Ackoff  Collaboratory  for  Advancement  of  the  Systems  Approach  (ACASA).  The 
focus  of  his  research  has  largely  been  on  aesthetic  and  cognitive  engineering  of  embedded  game-theoretic  agents 
that  can  help  humans  improve  their  learning,  performance,  and  systems  thinking  in  task-environments. 

According  to  Silverman,  a  holy  grail  for  military,  diplomatic,  and  intelligence  analysis  is  a  valid  set  of  software 
agent  models  that  act  as  the  desired  ethno-political  factions  so  that  one  can  test  the  effects  that  may  arise  from 
alternative  courses  of  action  in  different  lands.  This  presentation  enumerated  the  challenges  of  such  a  testbed  and 
described  best-of-breed  leader  and  follower  profiling  models  implemented  to  improve  the  realism  and  validity  of 
the  agent.  Realistic,  ‘descriptive’  agents  were  contrasted  to  rational  actor  theory  in  terms  of  the  different 
equilibria  one  would  expect  to  emerge  in  conflict  games.  These  predictions  were  examined  in  two  real  world 
cases  (Iraq  and  Southeast  Asia)  where  the  agent  models  were  subjected  to  validity  tests  and  a  policy  experiment 
was  then  run.  Silverman  concluded  by  arguing  that  substantial  effort  on  game  realism,  best-of-breed  social 
science  models,  and  agent  validation  efforts  is  essential  if  analytic  experiments  are  to  effectively  explore  conflicts 
and  alternative  ways  to  influence  outcomes.  Such  efforts  are  likely  to  improve  behavioral  game  theory  as  well. 

Thus  there  is  a  need  for  a  toolbox  (vice  a  single  tool).  Silverman  argued  the  need  to  cross-fertilize  across  the 
“silos”  of  social  science:  the  key  involves  “interactions.”  He  said  the  major  hole  in  our  knowledge  lies  at  the 
bottom:  social  science  theories.  As  a  technique,  Silverman  proposed  the  use  of  Performance  Moderator 
Functions  (PMF),  drawing  on  “best  of  breed”  PMFs  for  the  toolbox. 
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2.1.9  Ferleman 


Dr.  Thomas  Ferleman  is  an  Associate  with  Booz  Allen  Hamilton  and  is  the  Global  Futures  Modeling  & 
Simulation  (GFM&S)  practice  lead.  He  presented  “Modeling  Global  Futures:  Forecasting  Patterns,  Determining 
Alternatives,  and  Mitigating  Risk.” 

Ferleman  described  his  model  and  its  architecture.  The  architecture  draws  on  mix  of  domains  (e.g.,  economics, 
socio-political,  international  political,  population,  agriculture,  energy,  technology,  and  environment).  Ferleman 
exercised  the  model  for  a  variety  of  scenarios;  e.g.,  self-sustainment,  export,  human  development,  in  forecasting 
trends  over  several  decades.  In  his  presentation,  Ferleman  cited  the  work  of  Barry  Hughes,  University  of  Denver, 
as  a  major  source.  In  the  discussion  of  the  presentation,  several  issues  were  raised  about  the  experimental  design 
used  for  these  scenarios. 

2.2  DISCIPLINARY  WORKING  GROUPS 

The  workshop  included  four  disciplinary  working  groups:  micro-social  sciences,  macro-social  sciences, 
operations  research,  and  VV&A.  The  description  of  each  working  group  and  its  results  is  divided  into 
membership,  presentations,  discussion  topics,  and  expressed  needs  sections.  The  “expressed  needs”  are  those 
statements  from  the  working  group  that  most  directly  addressed  the  Workshop  purpose. 

2.2.1  Micro-Social  Sciences 

Membership: 

•  Jerrold  Post  (chair) 

•  Lawrence  A.  Kuznar  (co-chair) 

•  Myriam  Abramson 

•  Eli  Berman 

•  Deborah  Duong 

•  Lee  Ellen  Freedland 

•  Rebecca  Goolsby 

•  Jeff  Hansberger 

•  Lauren  Lee 

•  Will  Moore 

•  Jennifer  O’Connor 

•  Peter  Pham 

Presentations: 

Dr.  Lawrence  A.  Kuznar  is  Chief  Cultural  Sciences  Officer  for  National  Security  Innovations  (NSI).  Prior  to 
joining  NSI,  Dr.  Kuznar  was  a  professor  of  anthropology  at  Indiana  University — Purdue  University,  Fort  Wayne 
in  Fort  Wayne,  IN.  His  research  focused  on  decision  theory,  theories  of  conflict  and  terrorism,  and  computational 
modeling.  He  presented  “Bridging  the  Micro-Macro  Gulf  in  Social  Science  Research:  Military  and  Intelligence 
Applications.”  The  key  points  of  this  presentation  were  the  following: 

•  Micro  units  of  society  are  individuals 

•  Methodological  individualism  and  reductionism  do  not  obviate  appreciating  social  wholes 

•  Appreciation  of  emergent  social  phenomena  is  grounded  in  understanding  the  complex  interactions  of 
their  units 

•  Units  should  not  be  static;  people  are  not  static,  their  states  and  behaviors  change 

•  Aim  of  social  modeling  should  be  the  emergence  of  macro-phenomena;  these  are  the  sorts  of  phenomena 
of  most  national  security  concern  (riots,  revolutionary  movements,  insurgencies,  terrorist  networks,  state 
collapses,  genocides) 


•  Ultimate  models  will  have  emergent  social  phenomena  that  will  take  on  their  own  rules  of  interaction; 
modelers  are  far  from  this 

•  Models  must  be  evaluated  and  validated;  they  must  predict,  although  precise  predictions  are  not 
reasonable 

•  Over-calibration  can  lead  to  accuracy  and  precision  but  not  validity,  especially  for  near-term  phenomena 

•  General  long-term  trends  can,  ceteris  paribus,  be  predicted,  but  there  is  a  many-to-one  mapping  of  models 
to  such  phenomena 

•  Ensemble  computing,  in  which  model  behavior  is  explored  over  its  complex  parameter  space  is  a  more 
practical  and  responsible  means  of  judging  the  validity  of  a  model 

Dr.  Jerrold  M.  Post  (M.D.)  is  professor  of  psychiatry,  political  psychology,  and  international  affairs  and  director 
of  the  political  psychology  program  at  The  George  Washington  University.  Dr.  Post  presented  “When  Hatred  Is 
Bred  in  the  Bone:  Psycho-cultural  Foundations  of  Contemporary  Terrorism.” 

Post  stated  that  the  lay  public  often  considers  terrorists  to  be  crazed  fanatics;  however,  in  fact  terrorist  groups 
regularly  exclude  emotionally  disturbed  individuals  from  their  ranks — after  all,  they  represent  a  security  risk.  His 
comparative  research  on  the  psychology  of  terrorists  does  not  reveal  major  psychopathology,  rather  that  the 
common  characteristic  of  terrorists  is  their  normality.  Post  said  that  social  psychology  provides  the  most 
powerful  lens  through  which  to  examine  and  explain  terrorist  behavior,  emphasizing  group  psychology  and 
collective  identity,  not  individual  psychopathology.  The  importance  of  collective  identity  and  the  processes  of 
forming  and  transforming  collective  identities  cannot  be  overemphasized.  This  in  turn  emphasizes  the  socio¬ 
cultural  context,  which  determines  the  balance  between  collective  identity  and  individual  identity.  Post  then 
described  the  importance  of  distinguishing  the  particular  type  of  terrorism  that  one  might  encounter,  as  shown  in 
Figure  4. 


Figure  4,  Post’s  Typology  of  Terrorism 


Discussion  Topics: 

1.  Defining  Micro 

The  first  and  perhaps  most  important  issue  was  the  definition  of  “micro”  for  social  phenomena.  Everyone 
agreed  that  the  distinction  between  micro  and  macro  was  arbitrary.  After  discussing  different  small  social 
units  (individuals,  families,  terrorist  cells),  Dr.  Will  Moore  made  the  following  suggestion: 
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Definition:  Micro  social  units  are  defined  as  the  lowest  reducible  decision  making  units  of  a  society; 
micro  units  are  capable  of  making  their  own  decisions. 

After  some  discussion,  the  group  came  to  consensus  on  this  issue. 

2.  Prediction 

Another  key  issue  concerned  the  use  of  models  for  prediction.  Everyone  agreed  that  the  use  of  models  for 
precise  and  accurate  point  prediction  (prediction  with  a  capital  “P”)  was  misguided.  Instead,  the  group 
argued  that  models  should  be  used  for  forecasting,  increasing  understanding,  and  creating  plausible 
futures.  Dr.  Kuznar  noted  that  these  terms  were  not  well  defined  and  that  people  still  use  increases  in 
understanding  to  predict.  Dr.  Post  noted  that  increases  in  understanding,  the  creation  of  plausible  futures, 
etc.  were  more  like  prediction  with  a  small  “p,”  and  everyone  in  the  group  agreed  that  traditional  notions 
of  prediction  (point  prediction,  capital  “P”)  were  not  useful  for  modeling  social  phenomena. 

3.  Other  Issues 

Participants  were  concerned  with  general  modeling  issues  and  concerns  with  responding  to  government 
RFPs/executing  contract  work.  In  particular,  participants  wanted  better  guidance  form  the  government 
regarding  what  kinds  of  models  were  desired  and  what  sources  of  data  were  available. 

Related  to  the  issue  of  more  transparent  requirements  was  the  argument  that  models  need  to  address 
commander’s  intent,  field  operator’s  concerns,  as  well  as  academic  and  theoretical  constructs. 

Models  should  also  model  U.S.  actions  in  order  to  provide  a  more  complete  picture  of  the  situations 
modeled. 

One  issue  that  was  discussed  was  the  need  to  adhere  to  doctrine  in  framing  models  and  their  variables  and 
parameters.  While  some  military  folk  advocated  such  an  approach,  other  military  personnel  and  the  rest  of 
the  participants  argued  that  this  was  overly  restrictive  for  R&D  modeling. 

Eventually,  the  group  settled  upon  binning  their  concerns  and  recommendations  into  several  categories. 
These  categories  include:  scale/units  to  be  modeled,  data  issues,  modeling  approaches,  and 
variables/constructs  to  be  modeled.  Those  points  that  the  group  deemed  of  paramount  importance  are  in 
bold. 

Scale/Units 

•  Individuals  are  obvious  atomistic  units  of  societies. 

•  Groups:  Micro  unit  can  make  its  own  decisions;  lowest  reducible  decision  making  unit 

•  This  definition  can  apply  to  families,  tribal  units,  even  military  juntas  and  the  elite  leadership  of  a 
country. 

•  Also  consider  scale  across  time. 

•  The  increase  in  rogue  leaders  implies  that  individual  profiling  is  important. 

Data  Issues 

•  Models  should  address  what  data  exist,  and  also  what  does  not  exist?  Identify  gaps. 

•  Models  should  be  actually  empirically  based;  no  synthetic  data 

•  Need  to  address  validation  of  data 

•  How  are  data  tied  to  questions  asked? 

•  There  is  a  great  need  for  a  Common  Ontology/  Thesaurus  (cross  disciplinary/cross 
military/civilian/International  Community).  A  common  social  ontology  is  probably  a  bridge 
too  far,  given  the  differences  among  social  science  disciplines,  but  modelers  should  provide  a 
thesaurus  of  their  ontology  to  show  how  that  ontology  relates  to  others. 

•  Modelers  should  use  expertise  of  personnel  previously  deployed 
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•  Models  should  provide  both  Hindsight  (retrodiction)  AND  foresight  (prediction) 

Approaches 

•  There  is  a  need  for  transparency  on  how  to  decide  on  models 

•  Modelers  should  specify  assumptions  and  limitations  of  models  (for  all  micro  models,  best 
with  short  term  phenomena) 

•  Identify  missing  components/theories 

•  Include  Qualitative  and  Quantitative  models  and  bridge  them 

•  Hypothesis  testing/gaming/COA  analysis  is  important  for  DoD  modeling  efforts 

•  Dashboard  -  to  apply  a  variety  of  models  to  specific  problems,  models  do  not  necessarily  need 
to  be  combined,  but  could  be  used  in  a  dashboard  on  a  problem  or  data  set  to  see  which  models 
are  most  appropriate  and  work  best  on  a  specific  problem. 

•  A  micro  model  must  operate  in  an  operational  framework  of  macro  and  other  models 

•  Sources  of  empirical  data:  content  analysis,  polling,  ethnographies,  case  studies,  SMEs  need  to  be 
varied  and  well-specified 

•  Models  should  provide  feedback  across  emergent  levels  of  the  phenomena  they  model 

•  Evolution  of  preferences  (habituation)  of  micro  units  is  important 

•  Evolution  of  norms  in  models  is  also  important 

•  Impact  of  macro-variables  on  micro  phenomena  should  be  included  in  a  model  (e.g.  effect  of 
interest  rates  on  individual  decisions  to  buy  a  home) 

•  Need  to  address  commander’s  intent  vs.  field  operators  vs.  what  social  scientists  can  provide 

•  Need  to  include  models  of  ourselves  in  our  models  of  red  and  green  populations 

•  Models  should  be  validated  in  part  by  peer  review 

•  Include  strong  representation  of  social  scientists/interdisciplinary.  The  days  of  engineers 
including  only  token  social  scientists  should  be  over. 

•  Multiple  futures  exploration  vs.  prediction 

•  Service  oriented  architecture 

•  How  do  other  countries  use  social  scientists? 

Variables/Constructs  to  Consider  (very  incomplete  list) 

•  Legitimacy 

•  Shared  motive/intent 

•  Differentiate  between  what  people  say  and  what  people  do 

•  Genetic  and  biological  variables  relevant  to  human  behavior 

•  Psychological  variables 

•  Need  leadership  psychological  profiles,  especially  since  many  post-cold  war  conflicts  are 
precipitated  by  rogue  leaders;  can  be  focused  on  leader  predominant  societies 

•  Family  structure/dynamics 

•  Peer  influences 

•  Economic  variables 

•  Education 

•  Demography 

•  Social  networks 

•  Goals  (explicit  vs.  implicit,  private  vs.  public) 

•  Leading  indicators/intervening  variables  that  lead  up  to  ultimate  dependent  variables 

•  Every  model  is  context-dependent 

•  No  model  is  knowledge-free 

•  The  independent  variables  are  themselves  context-dependent. 

•  Requirements  for  models  can’t  be  driven  by  the  consumer. 
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Expressed  needs: 

•  Particular  questions 

o  If  the  military  does  a  task  (for  example,  give  water  in  an  SSTR  task),  what  are  the  implications 
from  a  societal  point  of  view? 
o  What  to  look  for  in  a  village  you  are  going  through? 
o  Understanding  should  include 

■  Legitimacy 

■  Motive/intent,  Goals  (explicit,  implicit,  public,  private) 

■  Leadership  profiles 

■  Family  structure/dynamics 

■  Peer  influences 

•  Supporting  research  needs 

o  Data,  what  exists?  What  doesn’t  exist? 
o  Data  needs  to  be  validated 

o  Common  terminology  -  common  ontology  is  too  restrictive 
o  Permanent  repository  of  rules/relationships  like  Cyc  for  the  social  sciences 

•  Model  architecture  needs 

o  How  to  connect  the  people  on  the  ground  with  the  social  scientists? 
o  Should  include  model  of  ourselves 

o  “Prediction”  should  be  replaced  with  “forecasting,”  with  emphasis  on  possible,  multiple  futures 
o  Should  include  feedback  between  micro  and  macro 
o  Model  assumptions  should  be  evident 
o  Need  quantitative  and  qualitative  models 
o  Validation  is  required,  in  part  by  peer  review 
o  Service  oriented  architecture 

2.2.2  Macro-Social  Sciences 
Membership: 

•  Richard  Lobban,  Anthropology,  Rhode  Island  College  (chair) 

•  Loren  Cobb,  Mathematics,  University  of  Colorado  at  Denver  (co-chair) 

•  Tim  Bacon 

•  Cherie  Beck 

•  Nathan  Bos 

•  Doug  Clark 

•  Jennifer  Edwards 

•  David  Garment 

•  Margaret  Hayes 

•  Jonathan  Jackson 

•  Pauline  Kusiak 

•  John  Lawson 

•  Michael  Manno 

•  Pat  McKenna 

•  Danielle  Miller 

•  Charlene  Milliken 

•  Brice  Nicholson 

•  Maureen  O’ Mara 

•  Patricia  Partnow 

•  David  Siegel 
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•  Barry  Silverman 

•  John  Sokolowski 

•  Peter  Tikuisis 

•  Mike  Vlahos 

•  YunaWong 

•  Paul  Works 

•  Bill  Young 

Presentations: 

Dr.  Loren  Cobb,  of  the  Mathematics  Department  of  the  University  of  Colorado  at  Denver,  presented  "The  Impact 
of  Social  Theory  on  Model  Development."  This  paper  focused  on  the  use  of  social  theory  in  quantitative 
simulations: 

•  A  social  theory  is  any  well-developed  and  coherent  set  of  ideas  that  de-scribes  the  structure  and 
functioning  of  society,  and  its  ills.  These  may  come  from  any  of  the  social  sciences  (psychology, 
anthropology,  sociology,  economics,  political  science,  jurisprudence,  linguistics),  or  from  history  or 
philosophy.  Social  theories  may  deal  with  raw  human  behavior,  or  with  social  ideas,  or  with  systems  of 
symbolic  meaning  employed  by  a  society.  They  may  be  structural  or  dynamic;  they  may  be  deterministic 
or  fuzzy  or  stochastic;  and  they  may  be  expressed  in  the  medium  of  mathematics,  diagrams,  or  ordinary 
language. 

•  Social  theory  comes  to  bear  upon  social  modeling  in  the  smallest  spatial  and  temporal  units  of  analysis.  In 
macro-scale  modeling  the  behavior  of  society  as  a  whole  is  built  up  out  of  how  variables  and  conditions 
change  in  these  smallest  units.  When  these  are  small  geographic  cells,  then  the  function  of  social  theory  is 
to  provide  or  motivate  equations  that  specify  the  rates  of  change  of  state  variables  within  each  cell,  as 
influenced  by  their  current  values  in  the  cell  and  in  all  nearest  neighbors. 

•  In  models  of  refugee  flow  during  political  crises,  as  in  the  Rwandan  terror  campaign  of  1994,  for 
example,  existing  social  theory  proved  to  be  almost  useless  in  predicting  when  people  would  leave  their 
homes,  in  which  direction  they  would  decide  to  go,  and  where  they  would  decide  to  stop.  In  the  actual 
event  about  one  million  Rwandan  refugees  ended  up  in  a  treeless,  foodless,  waterless  volcanic  plain  near 
Gomah,  in  the  Congo. 

•  In  social  epidemiology,  in  contrast,  the  mathematical  theory  developed  in  the  20th  century  does  a 
reasonably  good  job  of  predicting  the  flow  of  most  contagions  through  the  population-density  map  of 
contemporary  societies  of  all  types.  To  the  extent  that  innovations  and  ideas  can  be  modeled  as  social 
contagion,  this  theory  may  also  apply  to  the  flow  of  new  ideas  through  society. 

•  On  the  other  hand,  the  inter-generational  social  epidemiology  of  post-traumatic  stress  disorder  (PTSD) 
reveals  a  serious  gap  in  social  theory.  What  happens  to  the  institutions  of  society  when  its  individual 
members  have  been  affected  for  generations  with  epidemic  levels  of  PTSD?  Social  theory  should  be  able 
to  tell  us  whether  the  individual  psychological  characteristics  of  PTSD  (e.g.  hypervigilance,  simplified 
good  vs.  evil  cognitive  categories,  hair-trigger  violence,  and  apocalyptic  thought  patterns)  are  capable  of 
seeping  into  institutional  worldviews,  religions,  and  behaviors.  Unfortunately,  existing  social  theory  is 
completely  silent  on  this  topic. 

Dr.  Richard  Lobban,  of  the  Anthropology  Department  of  Rhode  Island  College,  presented  "Reality  Check: 
Marrying  Modeling  with  Empiricism:  the  case  and  context  of  Sudan."  The  major  variables  and  conclusions  of  the 
paper  were  the  following: 

•  Among  all  so-called  "failed  states",  Sudan  presents  a  case  that  is  more  than  interesting:  it  may  be  the 
iconic  case,  the  signature  case  that  contains  almost  every  dynamic  of  importance. 
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•  Most  serious  Sudanists  probably  agree  that  the  causes  of  the  Darfur  conflict  are  multi-factorial,  interactive 
and  changeable.  Likewise  the  solutions  are  equally  complex.  Some  of  the  variables  are  stronger  than 
others,  some  clusters  of  variables  can  be  more  important  than  single  variable. 

•  Historical:  Probably  the  strongest  single  means  to  understand  Darfur  are  the  patterns  that  are  derived  from 
its  very  long  history.  At  times  it  has  been  sovereign  and  independent;  it  has  been  strong  and  weak.  It  has 
central  and  peripheral.  It  has  made  and  broken  power  in  the  central  Sudan. 

•  Economic:  There  are  the  broad  economic  factors  whereby  material  bur-dens  of  Sudan  fall  heavily  against 
the  people  of  Darfur  and  the  redistribution  of  wealth  from  manufacture,  agricultural  and  expanding 
petroleum  production  only  very  slowly  reaches  this  province.  Disruption  of  the  agricultural  and 
commercial  life  of  Darfur  partially  set  the  stage  for  the  pre-sent  complex  conflicts. 

•  Political:  The  overarching  domestic  /  national  /  internal  dimensions  of  the  conflict  in  Darfur  are  political 
and  failures  of  governance  that  could  not  fairly  and  adequate  address  the  underlying  sources  of  conflicts. 
There  are  very  many  overlapping  and  intersecting  political  variables. 

•  Ecological  &  Demographic:  It  is  clear  that  this  conflict  has  emerged  at  the  present  in  the  context  of 
persistent  ecological  crisis  of  increased  desertification  and  lack  of  production  and  limited  grazing  lands 
among  the  pas-toral  and  agricultural  peoples.  There  is  a  clear  correlation  with  rainfall  shortages, 
population  concentrations  and  the  areas  of  the  highest  level  of  clashes  and  differentiated  violence. 

•  Natural  Resources:  Mostly  at  subsistence  production  and  small  scale  trade  the  natural  resources  of  Darfur 
are  limited  in  terms  of  cash  exports.  Other  resources  such  as  water  in  the  far  north,  oil  in  the  south  and 
uranium  in  the  west  are  possible  future  factors. 

•  Military:  There  are  many  purely  military  factors  in  this  situation  which  pits  the  irregular  and  now  split, 
Sudan  Liberation  Army  (SLA)  led  by  Abd  al-Wahid  Mohamed  Nur  and  the  other  faction  led  by  Minni 
Arku  Min-nawi,  and  the  Justice  and  Equality  Movement  (JEM)  led  by  Khalil  Ibrahim  which  also  recently 
splintered.  All  are  more  or  less  fighting  against  formal  elements  of  the  Sudan  military  led  by  General 
Omar  Beshir  and  the  ir-regular  militia  called  locally  the  janjaweed  camel  and  horse  riding  raiders  led  by 
Musa  Hilal  among  several  others.  The  extensive  regional  presence  of  small  arms  makes  it  easy  to  take  up 
arms.  Smuggling  and  trafficking  has  long  been  part  of  the  local  economy. 

•  Cultural  &  Linguistic:  Central  Sudanese  are  broadly  Arabic  speakers,  and  in  Darfur  Arabic  is  more  likely 
a  second  language  than  the  primary  language  at  home.  The  skilled  use  of  Arabic  is  emblematic  of  higher 
social  position;  the  awkward  use  of  the  language  can  be  twisted  to  imply  people  of  a  lesser  degree  of 
social  status. 

•  Racio-Ethnic  Dimensions:  The  painful  question  of  racio-ethnic  prejudice,  is  sometimes  rendered 
simplistically  as  “Arabs  vs.  Africans.” 

•  Religion:  The  majority  of  combatants  on  all  sides  of  this  conflict  are  Sunni  Muslims,  but  this 
commonality  has  not  curbed  the  bloodshed. 

•  Gender  and  the  Social  Order:  So  far,  most  victims  are  refugee  women  and  girls  who  are  intimidated  and 
systematically  terrorized  by  rape  re-ported  by  many  independent  accounts.  As  long  as  sexism  and  gender 
prejudice  are  deeply  rooted,  the  ease  to  overlook  gender  abuse  assist  in  worsening  the  conflicts  on  the 
planet.  As  hard  as  it  is  to  discuss  race  in  Sudan  it  is  equally  difficult  to  frame  a  debate  or  undertake 
negotiations  about  gender. 

•  Conclusions: 
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o  The  crisis  in  Darfur  is  multi-faceted  and  needs  serious  reflection  on  cultural,  historical, 
ecological,  religious,  political  and  racio-ethnic  variables 

o  Foreign  intervention  has  both  positive  and  negative  features,  but  the  best,  but  not  simple  hope  lies 
for  constructive  intervention  (if  not  too  little  and  too  late)  might  lie  with  the  AU  force. 

o  Presently  this  crisis  situation  is  serious  and  likely  getting  worse.  At  this  rate  the  human  dimension 
is  likely  worsening  and  the  chance  for  anyone  to  control  the  levers  of  power  in  Darfur  is 
declining. 

o  The  rebel  movements,  the  janjawed  and  Sudanese  political  players  all  have  substantial 
complexity  relative  to  their  own  orientations  and  objectives. 

o  There  is  broad  domestic  and  regional  linkage  of  this  conflict  to  other  issues  and  political  players 
in  and  especially,  out  of  Sudan  in  the  utilization  and  degree  of  concern  for  the  Darfur  crisis. 

Discussion  Topics: 

1.  What  are  the  special  needs  of  macro-scale  simulation  models?  In  what  form  should  a  social  theory  be 
written  to  have  the  greatest  utility  for  the  modeling  and  simulation  community? 

Several  participants  made  the  strong  point  that  we  as  a  community  need  to  step  back  from  the  general 
head-long  rush  to  quantification  and  mathematical  model-building,  in  order  to  examine  and  better 
understand  the  interior  cognitive  models  and  categories  at  work,  both  in  the  subjects  under  consideration 
and  —  of  equal  or  greater  importance  —  within  those  who  are  constructing  the  quantitative  models. 

The  point  was  made  that  the  individual  disciplines  of  social  science  do  not  share  a  common  terminology, 
and  that  we  therefore  need  to  establish  a  correspondence  between  the  concepts  (or  semantic  categories)  of 
each  discipline  with  all  others. 

In  addition,  there  are  special  needs  of  macro-scale  social  theory.  The  primary  users  are  policy-makers, 
politicians,  and  the  general  public,  each  of  whom  are  more  likely  to  use  homilies  as  their  preferred  or 
indeed  only  interior  model,  rather  than  any  form  of  quantitative  or  mathematical  model.  Those  who 
construct  or  use  quantitative  models  may  find  themselves  talking  to  an  empty  audience. 

In  regard  to  the  connection  between  micro  and  macro-scale  social  theory,  there  are  micro  motives  for 
macro  behavior  and  vice  versa  -  e.g.,  the  majority  concurs,  a  vocal  minority  disagrees. 

The  subject  turned  to  cultural  change,  and  it  was  observed  that  there  is  little  agreement  on  the 
fundamental  nature  of  cultural  change.  We  need  a  better  lexicon  for  change,  and  also  an  appreciation  of 
its  ontology.  In  particular,  we  tend  not  to  pay  attention  to  the  authenticity  of  culture  change,  yet  when 
change  is  imposed  from  outside  by  foreign  influence  it  is  often  perceived  as  inauthentic,  and  either 
rejected  or  transformed  into  something  more  acceptable  to  the  ambient  culture. 

Throughout  the  preceding  discussion  a  gulf  was  easily  visible  between  two  radically  different  cognitive 
and  analytical  styles:  that  of  the  quantitative  and  mathematical  modelers  on  one  hand,  and  that  of  post¬ 
modern  social  scientists  on  the  other.  At  times  it  seemed  as  though  each  nurtured  an  internal  cartoon-like 
caricature  of  the  other's  concepts,  models,  and  worldviews. 

2.  Are  failed/failing  states  just  a  normal  part  of  the  rise  and  fall  of  states?  If  so,  should  we  mostly  be 
discussing  how  to  manage  failing,  collapsing,  transforming  states,  with  or  without  a  power  vacuum? 

In  the  discussion  of  failed  states,  the  point  was  made  that  we  need  to  include  "the  search  for  identity"  in 
any  model  of  state  or  nation  formation.  Efforts  to  increase  the  social  and  political  integration  of  a  nascent 
state-like  unit  depend  upon  an  awareness  of  common  identity  among  the  people  involved,  and  also  upon 
the  level  of  consciousness,  in  the  sense  in  which  that  term  is  used  in  Spiral  Dynamics  (Beck,  2005). 
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There  is  an  emerging  point  of  contact  between  the  two  worlds  of  social  modeling  (quantitative  vs.  post¬ 
modern),  in  the  relatively  new  area  of  agent-based  models  that  seek  to  learn  what  specific  characteristics 
of  individual  agents  produce  the  coalescence  of  geographic  political  units. 

The  contemporary  process  of  globalization  was  cited  as  bringing  forth  an  ironic  weakening  of  traditional 
nation-states  at  the  same  time  as  it  encourages  non-state  and  proto-state  entities  to  greater  political 
integration. 

More  generally,  it  was  noted  that  in  many  contexts  it  may  be  more  important  to  create  a  model  of  the 
primary  actors  (people,  organizations,  institutions)  than  of  the  dynamics  of  the  variables  that  characterize 
the  context.  Within  actors,  it  is  of  primary  importance  to  model  the  different  cognitive/semantic  maps  that 
they  employ  to  understand  any  given  political  situation. 

Furthermore,  we  asked  which  should  be  primary  goal  of  a  modeling  effort:  the  elegance  or  the  complexity 
of  the  resulting  model  and  their  supporting  theories?  While  this  may  ultimately  come  down  to  individual 
preference,  it  may  also  be  one  of  the  defining  distinctions  between  the  mathematical  and  the  post-modern 
approaches  to  model  construction. 

There  was  contention  regarding  the  failure  of  states  -  that  it  is  the  normal  course  for  states  to  divide  or  be 
absorbed  from  time  to  time.  By  artificially  freezing  the  status  quo  (of  territorial  boundaries)  states  which 
would  ordinarily  be  absorbed  fester  in  failure  and  stasis. 

3.  For  predictive  models  of  conflict,  how  can  we  get  the  attention  of  the  right  people  for  early  attention 
before  expensive  intervention  becomes  necessary? 

Discussion  turned  to  the  question  of  what  institutional  changes  may  be  needed  if  the  modeling 
community,  broadly  construed,  is  to  have  any  palpable  impact  on  policies,  plans,  and  programs. 

It  was  observed  that  the  Theater  Security  Cooperation  Plans  (TSCPs)  maintained  by  each  Geographic 
Combatant  Commander  present  an  immediate  opportunity.  At  present  these  plans  are  seldom  tested  or 
developed  with  the  aid  of  models  contributed  by  the  community.  Perhaps  we  could  foster  the  creation  of 
teams  or  task  forces  which  advise  and  assist  those  who  must  develop  or  test  operational  plans  and 
policies,  using  models  that  have  passed  a  rigorous  validation  review  and  are  written  to  standards 
developed  by  the  community  for  social  science  input,  documentation,  and  use  of  best  practices. 

Given  the  multiplicity  of  approaches  to  modeling,  not  to  mention  of  the  social  theories  that  support  these 
models,  it  might  be  better  to  present  a  federation  of  modelers  for  any  given  policy  or  plan-testing  context. 
The  multi-model  approach  avoids  the  dangerous  trap  of  identifying  any  one  model  or  approach  as  the  one 
single  right  answer,  and  instead  relies  on  a  family  of  models  to  provide  a  rich  base  of  insights  with  which 
to  improve  policy  and  plan  development. 

This  collaborative  effort  among  heterogeneous  modeling  teams,  grouped  together  into  a  loose  federation 
of  modelers,  has  many  appropriate  uses: 

•  development  of  a  multi-modal  vision  of  any  situation, 

•  generation  of  insightful  multi-factor  commentary  and  evaluation,  and 

•  provision  of  content-rich  recommendations  for  plans  and  policies. 

Given  the  variety  of  serious  challenges  and  threats  that  the  entire  planet  will  be  facing  in  the  near  future, 
including  climate  change,  decreasing  supplies  of  fossil  fuels,  a  brittle  world  financial  and  economic 
system,  and  the  near  certainty  of  global  pandemics  of  contagious  disease,  it  will  behoove  the  modeling 
community  to  move  forward  with  some  urgency  towards  improved  standards  for  model  construction, 
greater  self-discipline,  consistent  use  of  the  entire  range  of  available  modeling  approaches,  and  better 
integration  of  social  scientists  in  the  model  construction. 

Other  discussion  points: 
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Social  science  models  should  not  be  viewed  as  definitive,  but  as  decision  aids  not  unlike  weather  forecasts.  They 
will  always  be  probabilistic  (not  deterministic)  and  should  be  screened  by  subject  matter  experts.  A  meta-analysis 
of  various  model  predictions  would  likely  yield  a  "safe"  but  not  necessarily  accurate  prediction.  Subject  matter 
experts  would  be  helpful  in  the  latter.  Despite  the  plethora  of  available  models  it  is  unlikely  that  all  significant 
variables  have  been  suitably  quantified,  such  as  leadership  statements,  radicalization,  low  level  lawlessness  or 
civil  disobedience,  etc. 

A  desire  was  expressed  on  several  occasions  for  a  “SimCountry”  application;  Cobb  shared  an  early  prototype  of 
same.  Criticisms:  obvious  over-extension  of  technology  not  yet  capable  of  simulating  reality  effectively.  - 
admission  by  Cobb  that  any  solid  results  are  about  twenty  years  off. . . 

Expressed  needs: 

•  Particular  questions 

o  Predict  refugee  flow  during  political  crises:  inadequate 

o  Predict  the  flow  of  contagions  through  the  population-density  map  of  a  society:  good 
o  Predict  effects  on  societies  institutions  of  long-term  epidemic  PTSD:  inadequate 
o  Predict  when  cultural  change  will  be  regarded  as  authentic 
o  Understand  failed  states 

■  History 

■  Economy 

■  Political  contents 

■  Ecological  and  demographic  factors 

■  Natural  resources,  both  for  internal  use  and  as  exports 

■  Military  and  paramilitary  forces 

■  Culture  and  linguistic  situation 

■  Racio-ethnic  dimensions 

■  Religious  factors 

■  Gender  and  social  order 

■  External  influences  on  all  of  the  above,  e.g.,  globalization 
o  Support  Theater  Security  Cooperation  Plans  (TSCPs) 

•  Supporting  research  needs 

o  Establish  common  terminology  among  the  social  sciences 
o  Understand  interior  cognitive  models 
o  Either 

■  construct  converters  of  mathematical  models  to  homilies  or 

■  Educate  policy-makers,  politicians,  general  public  in  mathematical  models 

•  Model  architecture  needs 

o  Support  multi-modal  vision  of  situations 
o  Generate  insightful  multi- factor  commentaty  and  evaluation 
o  Provide  content-rich  recommendations  for  plans  and  policies 
o  Provide  probabilistic  statements,  screened  by  subject  matter  experts 
o  Don’t  use  internal  political  boundaries,  but  use  cells  with  flows  among  cells 

2.2.3  Operations  Research 

Membership: 

•  Mike  McGinnis,  VMASC  Executive  Director  (chair) 

•  Bob  Sheldon,  Senior  Analyst,  Group  W  Inc,  (co-chair) 

•  Gary  Citrenbaum 

•  Skip  Cole 

•  Dave  Davis 


17 


•  Krista  Elefante 

•  Dipak  Gupta 

•  John  Hummel 

•  James  Morris 

•  MattNickens 

•  Jay  Persons 

•  Brett  Pierson 

•  Kevin  Roney 

•  Vince  Roske 

•  A1  Sciarretta 

•  A1  Sweetser 

•  Victor  Wiley 

•  Ben  Wise 

•  Charles  Worrell 

Presentations: 

Dr.  John  Sokolowski  (VMASC)  -  “Population  &  Social  Dynamics  Modeling” 

CDR  Brett  Pierson  (J8)  -  “Systems  Dynamics  Modeling” 

Lt.  Col.  Larlai  Guermantes  (USAF  Staff)  -  “Terrorism  &  CT  M&S” 

Mr.  Mike  Ottenberg  (OSD  PA&E  SAC)  -  “Wargaming  /  Gaming” 

Mr.  Jack  Jackson  (TRAC -Monterey)  -  “Counter-Insurgency  Modeling” 

MAJ  Jay  Persons  (TRAC-FLVN)  -  “Discussion  of  HSCB  and  OR/M&S/Analysis  (USA/USMC  background 
study)”  This  study  identified  160  key  issues/questions,  derived  14  decision  issues  and  56  required  analytical 
capabilities,  and  identified  35  gaps.  Of  these,  20  were  associated  with  soft  science  (behavioral)  and  34  gaps  were 
associated  with  data. 

Discussion  Topics: 

•  Types  and  Levels  of  Models 

o  Systems  dynamics  not  an  adaptive  modeling  environment  but  very  appropriate  for  high  level 
aggregated  cause  and  effect  modeling 

o  Agent  based  models  better  suited  for  modeling  complex  adaptive  behaviors 

•  Semantics,  Lexicon,  Taxonomy,  Standards,  and  Repository 

o  HSCB  modeling  requires  standards  and  definitions  of  common  terms  for  both  OR  and  social 
science  communities 

o  Need  a  resource  repository  for  HSCB  models  and  data 

o  Establish  /  identify  references  &  organizations  to  oversee  (provide)  validated  models  and  data 
similar  to  those  for  warfare  models  (e.g.,  Joint  Munitions  Effectiveness  Model) 

•  Improve  understanding  and  manage  expectations.  Key  groups:  HSCB  model  users  /  consumers.  For 
what  purposes:  prediction,  forecasts,  exploration  (branches/sequels),  and  discovery  &  exploration  of  black 
swans 

•  Alternate  /  Complementary  Approaches:  holistic  architecture  (level  to-be-decided)  versus 
vignette/scenario  driven.  Which  is  most  appropriate  for  deriving  needs  for  these  models? 

•  Standards  for  data  lexicon,  semantics,  and  ontology 

o  Must  include  metadata  &  HSCB  protocols 
o  Must  be  developed  within  the  real-world  context  of  the  model 

•  Decomposition  of  the  problem  domains 
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o  See  for  example  various  ‘candidate  decomposition’  of  the  IW  environment  presented  during  the 
session. 

o  HSCB  Model  Architecture  requires  flexibility  to  integrate  /  aggregate  domains;  e.g.,  integrate 
social-religion-political  domains  when  dealing  with  the  Muslim  world 

•  HSCB  Model  Review 

o  HSCB  modeling  would  benefit  from  inclusion  of  Bayesian  analysis  with  SMEs  to  take  into 
account  a  combined  scale/score 

o  Need  a  consistent  approach  to  dealing  with  SME  evaluations 

o  Need  to  identify  /  include  multi-disciplinary  &  cross-disciplinary  experts  when  conducting  model 
review 

•  Identify  output  metrics  relevant  to  make  decisions 

The  working  group  also  decided  it  would  have  been  helpful  if  we  had  developed  ahead  of  time  .... 

•  HSCB  modeling  frame  of  reference  to  provide  a  context  to  development  of  our  (OR-working  group)  set 
of  requirements 

•  Reference  problem/case  study/scenario/vignette  to  bound  the  scope  of  the  problem-space  which  would 
have  in-tum  helped  focus  discussions 

•  Included  interagency  involvement  and  attendance  at  this  workshop 

Workshop  II  Considerations:  Reset  the  working  groups  versus  keep  same  groups;  form  new  groups  based  on 
Workshop  II  agenda;  Invite  interagency  personnel;  Sequence  sessions  and  presentations  to  maximize  information 
flow  and  knowledge  generation  &  sharing. 

Expressed  needs: 

•  Particular  questions 

o  prediction,  forecasts,  exploration  (branches/sequels),  and  discovery  &  exploration  of  black  swans 
o  What  are  the  key  questions  HSCB  models  will  answer? 

■  Identify  enduring,  reoccurring  questions  that  arise  in  military  operations 

■  Partially  addressed  by  Army/USMC  IW  Study 

•  Supporting  research  needs 

o  What  are  the  HSCB  theories  that  drive  how  the  models  function? 

o  HSCB  modeling  requires  standards  and  definitions  of  common  terms  for  both  OR  and  social 
science  communities 

o  Standards  for  data  lexicon,  semantics,  and  ontology 

■  Must  include  metadata  &  HSCB  protocols 

■  Must  be  developed  within  the  real-world  context  of  the  model 

o  Knowledge,  data,  and  algorithms  that  account  for  the  effects  of  influencers  (operational  activities) 
on  the  attitude/behavior  of  the  civilian  population  based  on  ethnic,  tribal,  cultural,  religious,  and 
political  considerations 

o  Data  and  algorithms  that  translate  civilian  attitudes  into  levels  of  cooperation  with  friendly  forces 
and  result  in  corresponding  levels  of  HUMINT  provided  by  the  civilian  population 
o  Knowledge,  data,  and  algorithms  to  account  for  discrimination  between  civilian  and  adversary 
actors  based  on  presented  physical  and  behavioral  signature  (e.g.,  insurgent  in  civilian  clothing) 
o  Knowledge,  data,  algorithms  that  account  for  how  affiliations  and  support  for  other  actors  change 
based  on  the  application  of  influencers  (e.g.,  friendly  operations,  government  activities,  adversary 
operations) 

o  Data  and  algorithms  that  account  for  changes  in  target  audience  attitudes  caused  by  the 
application  of  PSYOPS 

o  Knowledge,  data,  and  algorithms  that  reflect  adversary  HUMINT  networks  (e.g.,  attributes  of  the 
HUMINT  network,  how  the  network  is  formed,  how  the  network  adjusts  if  a  node  or  element  is 
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removed,  what  adversary  activities  tend  to  facilitate  or  discourage  the  population’s  provision  of 
HUMINT) 

o  Knowledge,  data,  and  algorithms  to  account  for  unique  adversary  PSYOPS  techniques  and  the 
effect  of  those  techniques  on  the  target  audience 

o  Data  and  algorithms  to  represent  the  effects  of  CMO  on  the  attitudes  of  the  civilian  population  (  or 
other  target  audience) 

o  Data  to  implement  the  effects  of  essential  services  (or  lack  thereof)  on  civilian  population’s 
attitudes/behaviors 

o  Knowledge,  data  and  algorithms  accounting  for  the  effect  of  governmental  corruption  on 
governmental  institutions  and  on  the  civilian  population’s  attitudes/behaviors 
o  Knowledge,  data  and  algorithms  reflecting  the  attitudes/behavior  of  actors  (e.g.  civilian 
population)  based  on  the  state  of  physical  infrastructure  used  by  the  actors 
o  Knowledge,  data  and  algorithms  accounting  for  non-homogeneous  groups  of  actors.  This  is 
particularly  problematic  when  group  members  have  overlapping  affiliations  (e.g.  a  single  actor  or 
group  belongs  to  multiple  groups  -  religious  groups,  ethnic  groups,  political  groups,  tribal  groups, 
etc.) 

o  Data  and  algorithms  accounting  for  levels  of  civilian  support  for  and  the  provision  of  physical  or 
monetary  resources  to  adversaries 

o  Knowledge,  data  and  algorithms  accounting  for  the  performance/effectiveness  of  actor 

organizations  based  on  the  level  of  cooperation  between  those  organizations  (e.g.  USAID  and 
local  government) 

o  Knowledge,  data  and  algorithms  accounting  for  the  behavior  of  actors  (e.g.  civilian  population, 
religious  leaders)  based  on  their  level  of  support  for  the  existing  government 
o  Knowledge,  data  and  algorithms  accounting  for  the  legitimacy  of  the  existing  government  as 
viewed  from  outside  the  nation  by  external  groups  and  the  effect  of  international  legitimacy  on 
government  effectiveness 

o  Knowledge,  data  and  algorithms  accounting  for  the  of  the  state  of  the  existing  legal  system  and  its 
impact  on  the  attitudes/behaviors  of  the  civilian  population 
o  Data  to  define  civilian  attitudes/behaviors  based  on  existing  economic  conditions  and  how  the 
attitudes/behaviors  change  as  the  economic  conditions  change 
o  Knowledge  and  data  about  the  effects  of  media  activity  on  the  attitudes/behaviors  of  actors 
o  Knowledge  and  data  about  the  effects  of  friendly  operations  on  media  themes  and  activity 
Model  architecture  needs 

o  How  to  handle  proprietary  information? 

o  How  to  handle  “personal”  information  (privacy  and  use  of  information)? 
o  Need  a  resource  repository  for  HSCB  models  and  data 

o  Establish  /  identify  references  &  organizations  to  oversee  (provide)  validated  models  and  data 
similar  to  those  for  warfare  models  (e.g.,  Joint  Munitions  Effectiveness  Model) 
o  Concerns  related  to  VV&A 

■  How  do  modelers  reduce  uncertainty  in  “squishy”  HSCB  models? 

■  How  much  uncertainty  reduction  is  feasible/sufficient? 

■  Subjective  nature  of  models  suggests  data  will  be  required  for  both  V&V  and  to  train  the 
model? 

o  holistic  architecture  (level  TBD)  versus  vignette/scenario  driven.  Which  is  most  appropriate  for 
deriving  needs  for  these  models? 

o  HSCB  Model  Architecture  requires  flexibility  to  integrate  /  aggregate  domains;  e.g.,  integrate 
social-religion-political  domains  when  dealing  with  the  Muslim  world 
o  HSCB  Model  Review 

■  HSCB  modeling  would  benefit  from  inclusion  of  Bayesian  analysis  with  SMEs  to  take 
into  account  a  combined  scale/score 

■  Need  a  consistent  approach  to  dealing  with  SME  evaluations 
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■  Need  to  identify  /  include  multi-disciplinary  &  cross-disciplinary  experts  when 
conducting  model  review 

o  Identify  output  metrics  relevant  to  make  decisions 

o  Tailorable  and  adaptable  HSCB  models 

■  Focus  on  identification  of  common  factors  as  a  baseline  for  FISCB  models  (e.g., 
ODU/VMASC  Insurgency  Studies  of  Columbia  and  Nigeria  found  60  of  125  insurgency 
factors  to  be  common) 

■  Easily  change/add  non-common  factors 

o  Define  /  design  a  family  (framework)  of  HSCB  models  that  can  be  integrated  and  federated  to 
model  specific  scenarios  and  casual  effects 

o  Adopt  a  model  framework  that  can  accommodate  meta-model  and  meta-data  aggregation  and 
disaggregation 

o  To  make  models  efficient  &  cost  effective.  Minimize  model  development  costs,  runtime  and 
overhead  (admin,  user,  and  developer). 

o  To  replicate  real  world  activities  and  instantiate  HSBC  theory/protocols/methods 

o  To  represent  ‘external’  influences  (e.g.,  ever-changing  views/norms,  local/societal  demographics, 
and  processes  of  individuals  and  societies)  that  in  turn  influence  both  physical  and  cognitive 
environments. 


2.2.4  VV&A 
Membership: 

•  Dean  S.  Hartley  III,  Principal,  Hartley  Consulting  (chair) 

•  Marina  Arbetman-Rabinowitz,  Sentia  Group  and  the  Claremont  Graduate  University,  (co-chair) 

•  Michael  Bailey 

•  Pauline  Baker 

•  Chuck  Barry 

•  Todd  Brethauer 

•  Alok  Chaturvedi 

•  Claudio  Cioffi-Revilla 

•  Rick  Cunningham 

•  Jack  Goldstone 

•  Ari  Greenberg 

•  J.C.  Herz 

•  Charles  Macal 

•  Brett  Marvin 

•  Mary  McDonald 

•  Jimmie  McEver 

•  Anne  McGee 

•  Lisa  Moya 

•  Jennifer  O’Connor 

•  Maureen  O’ Mara 

•  Michael  Ottenberg 

•  Brett  Pierson 

•  Eunice  Santos 

•  Gary  W.  Schaeff 

•  Stephen  Shellman 

•  V.S.  Suhrahmanian 

•  Ben  Wise 
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•  Jordan  Willcox 

•  Ted  Woodcock 

Presentations: 

Dr.  Dean  S.  Hartley  III  is  the  Principal  of  Hartley  Consulting  and  has  done  work  in  VV&A  for  35  years.  Hartley 
presented  the  paper,  “VV&A  for  DIME/PMES1I  Models.” 

Verification  and  validation  (V&V)  are  difficult  processes  and  the  complexity  of  the  DIME/PMESII  (diplomatic, 
information,  military,  economic/political,  military,  economic,  social,  information,  infrastructure)  domain  adds  to 
the  difficulty.  However,  V&V,  under  the  proper  approach  is  not  impossible  and  supports  the  need  for 
accreditation  (the  “A”  in  VV&A).  This  paper  describes  a  risk-based,  entrenched  methodology  that  meets  the 
needs  of  DIME/PMESII  modeling.  The  paper  also  covers  the  special  cases  of  agent-based  models  and 
compressed  and  hyper-compressed  VV&A. 

Dr.  Marina  Arbetman-Rabinowitz,  of  the  Sentia  Group  and  the  Claremont  Graduate  University,  presented  the 
paper,  “Time  to  Measure  Up:  Dirty  Little  Secrets  in  Data  Collection  Or  on  how  Econometrics  is  Easier  without 
data.” 

According  to  Arbetman-Rabinowitz,  in  political  economy  there  are  an  abundance  of  theoretical  postulates  and 
models,  yet  very  few  indicators  and  measures  are  developed  to  test  the  validity  and  reliability  of  the  arguments 
beyond  contrasting  the  results  to  reality.  She  said  this  is  particularly  evident  in  the  lack  of  value  placed  on  data 
collections,  with  little  to  no  emphasis  attached  to  maintaining  time  series  data  bases  that  relate  to  variables  beyond 
the  economic  realm.  Modem  data-collection  techniques  allow  us  to  systematically  collect  information  about  our 
objects  of  study  (people,  objects,  phenomena)  and  about  the  settings  in  which  they  occur.  The  constraints  in 
availability  of  data  are  one  reason  why  data  is  collected  haphazardly;  and  as  a  consequence  makes  it  difficult  to 
answer  our  research  questions  in  a  conclusive  manner.  The  presentation  charted  the  steps  to  successful  data 
collections  through  specification  before  estimation,  operationalization,  data  collection  techniques,  data  collection 
and  measurement  repeatability,  reproducibility,  accuracy,  and  stability. 

Often  the  Failure  of  Predictive  Models  is  due  to  inaccurate  specification  and  poor  data  collection: 

•  Fundamentals  of  applied  work  require  a  problem  oriented  approach  rather  than  technique  oriented 
approach 

•  Hierarchical  Modeling:  The  organizing  question  should  navigate  across  theories,  models  and  levels  of 
analysis 

•  Specifications  of  variables:  require  Concepts  and  Operationalization  (semantics  and  ontology) 

•  Inventory  of  concepts,  variables  and  specifications  is  needed 

•  Data  Integrity,  Consistency,  Maintenance,  and  Dissemination  are  required 

•  Reliability  and  Validity  Tests  of  operational  qualitative  and  quantitative  variables  must  be  performed 

Discussion  Topics: 

The  following  were  the  formal  discussion  topics: 

•  Given  the  scope  and  breadth  and  varying  depths  of  the  methodologies  (theories  and  computational 
modeling  techniques),  tools  (models  and  systems  of  models),  data  related  issues  and  econometric 
techniques  (time  series,  static,  real-time) 

o  How  should  we  organize  these  dimensions  to  make  them  comprehensible  in  a  useful  way? 
o  Assuming  VV&A  can  help  us  with  this  organization  by  structuring  what  we  know  about  them, 
how  do  we  make  that  happen? 

o  What  do  we  want  from  HSCB  models,  annotated  by  “wishful  thinking,”  “possible  in  the  future,” 
“possible  in  the  near  future,”  and  “possible  now”? 

•  Data  Requirements  -  Has  this  been  a  limiting  factor? 

•  How  much  time  is  allocated  in  your  projects  to  data  related  items? 
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•  How  can  data  help  improve  your  efforts?  What  are  the  gaps?  Ideal  world? 

The  group  performed  two  brainstorming  efforts.  The  first  addressed  the  question  of  organizing  the 
methodologies,  tools,  data,  and  techniques.  The  consensus  was  that  V&V  should  be  used  to  create  tags  that 
define  the  items  to  a  much  greater  depth  than  is  usually  found  in  model  catalogs  and  that  a  visualization  tool 
should  be  created  to  allow  a  user  to  search  and  identify  items  that  meet  the  user’s  criteria.  The  group  created  an 
initial  list  of  tags,  as  follows: 

•  General  characteristics 

o  Systemic  Organization  -  Inputs,  Processes,  Outputs 
o  Academic  Discipline 
o  Function  -  Outcomes  produced 
o  Assumptions  Used 
o  Level  (strategic,  operational,  tactical) 
o  Domains:  land,  sea,  air,  space,  cyberspace 
o  Time  -  Forecasting,  Assessment,  Historical 
o  Granularity:  Sub-individual,  Individual,  Group, 
o  Coverage  of  DIME/PMESII 
o  Methodological  Approach 

o  Representation:  Descriptive,  Observation,  Intervention 

•  Secondary  characteristics 

o  Update  rates  on  Data 
o  Update  to  the  Model  (granularity  in  time) 
o  Military  Functions,  maneuver,  fires,  C2,  intel,  sustainment 
o  Surfboards:  training,  analysis,  T&E,  planning,  operations 
o  #  of  sides  -  where  can  model  be  used  .. 

o  Deterministic  vs.  stochastic,  closed  form  vs.  human  in  the  loop, 
o  Pedigree-  metadata  data  -  characteristics  of  data 

•  User  Needs  characteristics 

o  Owner  /  How  do  I  acquire  it  (COTS,  MOTS,  web  services) 
o  Ease  of  use 

o  What  is  time  /  effort  /  cost  of  usage 
o  Maintenance  (software  &  training)  requirements 
o  Interoperability  with  other  systems  (DoD,  agencies,  allies) 
o  Tactically  mobile,  reliable,  networked 

The  second  brainstorming  effort  created  a  set  of  possible  end-user  questions  that  HSCB  models  would  need  to 
address.  The  group  was  not  able  to  define  how  easily  these  questions  could  be  answered.  Following  the 
workshop,  a  number  of  participants  emailed  their  responses.  These  were  collated  and  are  shown  in  Figure  5. 
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Why  people  become  insurgents? 

2 

Why  are  people  X  becoming  Insurgent  Y  in  location  Z? 

3 

Which  of  these  people  are  becoming  insurgents? 

3 

How  do  I  “reverse  this”?  How  long?  What  will  it  take? 

6 

How  are  the  red  team  going  to  respond  to  our  actions?  Who  is  the  red  team? 

5 

Given  limited  $$,  what  part  of  DIME  should  we  invested  in? 

4 

Given  desire  for  success,  which  COA  should  be  done? 

3 

Tools  that  provide  situational  awareness  and  short-term  forecasting.  (Desired) 

1 

What  are  the  consequences  including  unintended  of  our  actions? 

4 

How  robust  are  results  against  uncertainty?  Model  incompleteness  -  other  possible  answers. 

5 

What  are  the  system  drivers  /  interaction  mechanisms  of  the  results. 

3 

Scalable  organizational  performance  model  of  social  institutions  [city/town].  (MPICE). 

2 

Model  that  accommodates  all  theories,  or  knows  why  not. 

6 

Bridge  training/education/operations  -  all  tailored  to  be  the  same,  see  book  “Ender’s  Game”. 

7 

Do  we  want  a  GIG  (global  information  grid)  service  of  DIME/PMESII.  We  want  something  that 
plugs  in  a  question  and  spits  out  an  answer  relative  to  the  population. 

Technical- 

models  should  measure  outcomes  and  not  inputs.  Not  by  what  you  do,  but  by  what  happens 
when  you  do  it. 

3 

Standardized,  interoperable  data.  Needs  to  be  available  on  point  to  the  end  user  and 
automatically  enforced.  Knowledge  of  what  model  does  not  do. 

5 

Figure  5.  Estimates  of  Number  of  Years  to  Solve  Needs 


There  was  not  enough  time  to  fully  discuss  the  questions  on  data  gathering  during  the  workshop.  The  questions 
were  emailed  to  the  participants  and  the  responses  are  shown  in  Figure  6. 
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Have  data  requirements  been  a  limiting  factor?  (1=  very,  10=none) 

1-2 

3-4 

5-6 

7-8 

9-10 

How  much  time  is  allocated  to  your  projects  for  data  related  items? 

0-10% 

11-20% 

our  problem  is  not  as  much  with  the  lack  of  time  devoted  to  trying  to  find  data/information  . . .  more  with  the 
difficulty  of  finding  what  can  support  the  modeling  need 

21-30% 

Problem  of  incomplete  time  series  and  lack  of  correspondence  between  concepts  and  available  data 

31-40% 

41-50% 

How  can  data  help  improve  your  efforts?  What  are  the  gaps?  Ideal  world? 

Real  world  data  can  help  us  better  select  the  relationships,  functional  forms,  and  parameters  we  use  in  our  modeling  efforts.  The  gap  is  that  information  is 
hard  to  find,  or  when  found,  must  go  through  a  non-trivial  ‘conversion  or  translation  process’  to  be  usable  by  certain  models.  Ideal  world  is  that  all  of  the 
different  USG  projects  collecting  HSCB-type  information  have  a  link  on  some  common  ‘portal’  that  USG  (or  their  contractors)  can  access  —  may  be 
classified. 

Especially  need  baseline  socio- cultural  data  in  areas  of  operations  before  US  intervention/occupation  so  that  ?  /t  can  be  measured. 

Economic  data  is  the  most  accessible,  therefore  social  concepts  get  translated  into  economic  variables  with  too  many  degrees  of  separation.  Need  a  clear 
ontology  of  concepts  across  social  scientists. 

We  need  an  overall  community  data  creation  effort.  There  is  a  methodology  for  building  a  library  of  these  taxonomies  to  create  standards  for  building 
datasets,  but  it  would  be  helpful  to  better  understand  the  scale,  range,  metrics  of  needed  data. 

Figure  6.  Impact  of  Data  Gathering 


During  the  general  discussion,  a  number  of  points  were  made  that  have  been  collected  and  organized: 

•  VV&A  issues 

o  Definitions 

■  Verification  is  the  process  of  determining  that  a  model  or  simulation  implementation 
accurately  represents  the  developer's  conceptual  description  and  specification 

■  Validation  is  the  process  of  determining  the  degree  to  which  a  model  or  simulation  is  an 
accurate  representation  of  the  real-world  from  the  perspective  of  the  intended  uses  of  the 
model  or  simulation. 

■  Accreditation  is  an  official  determination  that  a  model  is  acceptable  for  a  specific  purpose 
o  Actually  doing  VV&A 

o  Standards  of  accomplishment 
o  Agent-based  models 

o  Theology:  are  precise  definitions,  levels  of  accomplishment,  descriptions  of  pitfalls  more 
important  than  getting  started  with  performing  VV&A? 

•  Data  VV&A  issues 

o  DIME/PMESII  data  V&V  has  similarities  to  combat  model  data  V&V;  however,  significantly 
larger  complexity  of  the  data  sets  means  the  V&V  is  done  at  the  variable  level  by  [  time  span  & 
country  (or  unit  of  analysis),  conceptual  definition] 
o  [Standard/original  source]  Data  sets  are  never  really  ready  for  use  -  require  clean  up  to  fit  current 
need 

o  Need  Central  Library  with  concepts,  variables,  metadata  (Data  dictionaries,  semantic 

descriptions,  ontologies),  granularity  &  validation,  needs  to  include  qualitative  data  as  well  as 
quantitative  data 
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o  A  concepts  to  measurements  workshop  would  be  useful 

■  Outcomes  vs.  outputs  [counting  #  of  flyers  distributed  is  not  same  as  measuring  the 
impact  of  the  information  operation] 

•  Theories  &  methods  need  V&V  equivalent,  just  as  have  need  for  VV&A  of  data  &  tools 

o  In  the  sense  of  detailed  “examination”  of  same  to  produce  “tags” 
o  Need  standards  for  the  tags 

•  Lack  of  data 

o  Wealth  of  models 

■  No  data  or  insufficiently  “good”  data  to  test  them 
o  How  to  look  at  what  is  available,  what  is  not  available  - 

■  create  synthetic  data?  Anonymous  data? 

■  Who  can  get  best  data  -  not  necessarily  the  classified  data 
o  Database  maintenance  &  poor  dissemination  of  data  &  metadata 

o  Models  that  use  “similar”  data  to  the  data  that  are  really  needed  for  the  model  may  have 

dissimilar  outcomes  from  what  they  ought  to  have  -  because  there  is  an  additional  implicit  model 
of  the  real  data  by  the  “similar”  data 

■  This  is  a  thing  that  VV&A  needs  to  look  for 

•  Legal  issues 

o  Data  Ownereship 

■  Who  owns  data/intellectual  property 

■  If  the  government  owns  it,  it  may  be  subject  to  FOIA  release 

■  It  may  be  illegal  for  the  government  to  own  or  even  store  some  data 

■  Government  sponsored  projects  and  attendant  data  restrictions  and  requirements 
o  Consequences 

■  What  happens  when  data  collected  for  one  purpose  (passive)  is  made  available  to  support 
US  military  operations?  Are  original  (non-military)  researchers  potentially  liable? 

■  What  happens  when  incorrect  data  is  used  to  formulate  policy  that  harms  innocent 
individuals? 

■  Lawyers  to  keep  you  from  being  sued  when  someone  you  interviewed  is  killed 

■  FISA  and  the  Telecom  companies 
o  Distribution  restrictions 

■  Classification 

■  Academic  retention  until  publication 

■  Privacy  issues 

•  The  want  is  for  data  vs.  model  co-evolution 

•  Deciding  how  close  we  are  to  being  able  to  do  X  (answering  a  particular  question)  is  hard 

o  We  can  almost  always  build  a  model  that  purports  to  do  X 
o  How  do  we  know  if  it  really  does  X? 
o  How  do  we  know  when  we  will  be  able  to  do  it  in  future? 

o  What  data  are  needed  to  do  testing  of  whether  X  is  done?  Time  &  money  are  involved  in  getting 
the  data. 

•  Added  Value  of  Workshop:  Opening  of  Communications 

o  People  come  up  afterwards  and  say  -  yes  (or  no) 

o  Working  Group  is  sending  out  to  group  for  post-workshop  feedback:  Group  could  not  agree  on 
whether  these  some  could  be  answered. 

The  group  also  created  a  set  of  recommendations  to  NDU: 

•  Define  Data 

o  Start  with  PITF  data  gathering  project  and  extant  data  list 
o  Consider  whether  HSCB  modeling  requires  a  larger  data  set 
o  Define  metadata  requirements 
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o  Parcel  out  conceptual  islands  for  validation  of  concepts-measurements 
o  Gather  and  analyze  data 

■  Commission  searches  through  social  sciences  professional  societies 

■  Hold  a  conference  to  refine  metadata  and  data  V&V 

■  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging 
purposes 

•  Define  Theories 

o  Start  with  extant  theory  list 
o  Define  metadata  requirements 
o  Gather  and  analyze  theories  to  models 

■  Commission  searches  through  social  sciences  professional  societies 

■  Hold  a  conference  to  refine  metadata  and  theory  V&V 

■  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging 
purposes 

•  Define  Methods 

o  Start  with  the  Encyclopedia  of  Operations  Research  and  Management  Science 
o  Define  metadata  requirements 
o  Gather  and  analyze  methods 

■  Commission  searches  through  INFORMS  and  MORS 

■  Hold  a  conference  to  refine  metadata  and  method  V&V 

■  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging 
purposes 

•  Define  Models 

o  Start  with  the  extant  list  of  DIME/PMESII  models  (Hartley,  NRL,  JFCOM,  OSD  PA&E) 
o  Define  metadata  requirements  (start  with  DIME/PMESII  Model  VV&A  Tool  and  above  sources) 
o  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging  purposes 

•  Hold  a  Data  Library  Concept  Workshop  (after  above  work) 

o  Consider  the  potential  problems  of  a  Data  Library 

o  Consider  the  options:  central  library,  virtual  library  (distributed),  government  vs  academic  vs 
joint 

o  Consider  the  legal  issues 

o  Consider  the  data  maintenance,  creation,  VV&A,  tagging,  and  funding  issues 

•  Hold  a  Measures  of  Merit  (MOMs)  Workshop  (after  first  four  items) 

o  Outcomes  vs  outputs  vs  inputs  measures  [counting  #  of  flyers  distributed  is  not  same  as 
measuring  the  impact  of  the  information  operation] 
o  Desired  MOMs  vs  availability  of  data  vs  model  conversion  of  data  to  MOMs 

Expressed  needs: 

•  Particular  questions 

o  Why  people  become  insurgents? 

■  Why  are  people  X  becoming  Insurgent  Y  in  location  Z? 

■  Which  of  these  people  are  becoming  insurgents? 

■  How  do  I  “reverse  this”?  How  long?  What  will  it  take? 

■  How  are  the  red  team  going  to  respond  to  our  actions?  Who  is  the  red  team? 
o  Given  limited  $$,  what  part  of  DIME  should  we  invest  in? 

o  Given  desire  for  success,  which  COA  should  be  done? 
o  Tools  that  provide  situational  awareness  and  short-term  forecasting.  (Desired) 
o  What  are  the  consequences  including  unintended  of  our  actions? 

o  How  robust  are  results  against  uncertainty?  Model  incompleteness  -  other  possible  answers, 
o  What  are  the  system  drivers  /  interaction  mechanisms  of  the  results, 
o  Scalable  organizational  performance  model  of  social  institutions  [city/town].  (MPICE). 


27 


o  Model  that  accommodates  all  theories,  or  knows  why  not. 

•  Supporting  research  needs 

o  Central  Library  with  concepts,  variables,  metadata  (Data  dictionaries,  semantic  descriptions, 
ontologies),  granularity  &  validation,  needs  to  include  qualitative  data  as  well  as  quantitative  data 
o  Data  Integrity,  Consistency,  Maintenance,  and  Dissemination  are  required 
o  Reliability  and  Validity  Tests  of  operational  qualitative  and  quantitative  variables  must  be 
performed 

o  Legal  issues  regarding  data  collection  and  storage  need  to  be  identified 
o  Concepts  to  outcomes  (measures  of  merit)  connections 

•  Model  architecture  needs 

o  Bridge  training/education/operations  -  all  tailored  to  be  the  same,  see  book  “Ender’s  Game”, 
o  Do  we  want  a  GIG  (global  information  grid)  service  of  DIME/PMESII.  We  want  something  that 
plugs  in  a  question  and  spits  out  an  answer  relative  to  the  population, 
o  models  should  measure  outcomes  and  not  inputs.  Not  by  what  you  do,  but  by  what  happens  when 
you  do  it. 

o  Standardized,  interoperable  data.  Needs  to  be  available  on  point  to  the  end  user  and  automatically 
enforced.  Knowledge  of  what  model  does  not  do. 
o  Models,  data,  methods,  &  theories  all  need  to  be  characterized  with  tags  (done  with  a  type  of 
V&V),  with  visualization  tool  to  find  desired  entries 
o  Hierarchical  modeling  to  navigate  across  theories,  models  and  levels  of  analysis 
o  Data  and  models  should  co-evolve 
o  Best  practices  guide  for  VV&A  of  models  and  data 

2.3  PROBLEM  DOMAIN  WORKING  GROUPS 

The  workshop  included  four  problem  domain  working  groups:  deterrence,  counterinsurgency,  counter-terrorism, 
and  SSTRO.  The  description  of  each  working  group  and  its  results  is  divided  into  membership,  presentations, 
discussion  topics,  and  expressed  needs  sections.  The  “expressed  needs”  are  those  statements  from  the  working 
group  that  most  directly  addressed  the  Workshop  purpose. 

2.3.1  Deterrence 

Membership: 

•  Pat  McKenna,  STRATCOM  (chair) 

•  Yuna  Wong,  OSD  (co-chair) 

•  Myriam  Abramson 

•  Katherine  Banko 

•  Loren  Cobb 

•  Skip  Cole 

•  Max  Crownover 

•  Ivy  Estabrooke 

•  Richard  Hayes 

•  Krista  Hendry 

•  Jonathan  Jackson 

•  Richard  Lobban 

•  Charles  Macal 

•  Robin  Marling 

•  James  Morris 

•  Jennifer  O’Connor 
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•  Eunice  Santos 

•  David  Siegel 

•  John  Sokolowski 

•  V.S.  Subrahmanian 

•  Peter  Tikuisis 

•  Jordan  Wilcox 

•  Bill  Young 

Presentations: 

Mr.  Pat  McKenna  is  the  Senior  Advisor  -  Analysis,  STRATCOM,  Plans  and  Policy  Directorate.  His  current 
focus  is  developing  and  implementing  approaches  to  enhance  the  analytic  foundations  of  the  command's  plans. 
Specific  activities  include  deterrence  and  PMESII  assessment  capabilities.  McKenna  presented  “The  Deterrence 
Analytic  Challenge.” 

According  to  McKenna,  the  analytic  processes  used  for  Assurance/Dissuasion/Deterrence  (ADD)  have  changed 
little  since  the  Cold  War,  when  defense  analysts  focused  on  mutual  deterrence  between  two  nuclear-armed 
adversaries.  Since  deterrence  proved  successful,  and  bi-polar  competition  remained  relatively  stable  for  decades, 
each  government  began  to  believe  that  it  understood  the  other's  values  and  motives.  Today,  we  are  faced  with 
different  types  of  ADD  and  policy  analysis.  Our  actions  and  policies  target,  or  indirectly  affect,  entities  that  may 
or  may  not  be  legitimate  governments,  may  or  may  not  be  conventional  military  powers,  and  may  or  may  not 
share  our  cultural  norms.  The  world  is  now  a  multi-polar  environment,  where  our  objectives  against  one  player 
may  be  achieved  or  thwarted  by  our  actions  against  another.  Our  deterrent  actions  include  not  only  threats  of  cost 
imposition,  but  also  the  promise  of  benefit  denial  and  incentives  for  restraint.  The  effects  of  actions  propagate 
through  other  players  whose  strategic  interests  are  differently  aligned,  making  the  net  effects  difficult  to 
understand  beforehand  or  measure  afterwards. 

McKenna  continued  with  an  analysis  of  the  elements  of  deterrence  and  the  nature  of  the  problem,  which  was 
labeled  a  “wicked  problem.”  (The  definition  was  included,  but  basically  came  down  to  a  “really  bad”  problem.) 
McKenna  listed  Operations  Research,  game  theoretic,  and  social  sciences  techniques  that  are  applicable  to  the 
problem. 

Deterrence  Central  Idea  (from  DO  JOC  exec  summary) 

The  central  idea  of  the  DO  JOC  is  to  decisively  influence  the  adversary’s  decision-making  calculus  in  order 
to  prevent  hostile  actions  against  US  vital  interests.  This  is  the  “end”  or  objective  of  joint  operations 
designed  to  achieve  deterrence. 

An  adversary’s  deterrence  decision  calculus  focuses  on  their  perception  of  three  primary  elements: 

•  The  benefits  of  a  course  of  action. 

•  The  costs  of  a  course  of  action. 

•  The  consequences  of  restraint  (i.e.,  costs  and  benefits  of  not  taking  the  course  of  action  we  seek  to 
deter). 

Joint  military  operations  and  activities  contribute  to  the  “end”  of  deterrence  by  affecting  the  adversary’s 
decision  calculus  elements  in  three  “ways”: 

•  Deny  Benefits. 

•  Impose  Costs. 

•  Encourage  Adversary  Restraint. 

The  “ways”  are  a  framework  for  implementing  effective  deterrence  operations. 

Problem  Overview 

•  Deter  adversary  X  from  doing  Y  under  Z  conditions 


29 


•  Three  broad  areas  of  assessment 

o  Foundational  elements  (or  building  the  baseline  understanding  of  the  adversary) 
o  Pre-action  assessment  (or  deterrence  planning) 

o  Post  action  assessment  (or  examining  the  effect  of  an  executed  action) 

•  Complicated  by 

o  Uncertainty 

o  Conflicting  theories  and  approaches 
o  Unknown  (really  not  knowable)  deterrence  threshold 
o  Nth  order  effects 

•  Applicable  methodologies  and  tool  will  vary  by  X,  Y,  and  Z  as  well  as  area  of  analysis 

Dr.  Yuna  Fluh  Wong  is  a  Senior  Studies  analyst  for  SAIC  supporting  OSD  PA&E  Joint  Data  Support  (JDS).  She 
is  the  lead  JDS  analyst  for  irregular  warfare  (IW)  and  human  behavior  representation  (HBR).  Wong  presented  the 
paper  “A  Methodology  for  Valid,  Fast,  Cheap,  and  Practical  Analysis  of  Irregular  Warfare  Using  War  Games 
with  Computational  Adjudication  and  Analysis  Tools.” 

Her  paper  described  the  extended  war  game,  or  X-game,  which  is  the  OSD/PA&E  political  military  game  with 
extended  (week  long)  turns  to  allow  for  extensive  adjudication.  It  was  applied  to  the  Global  War  on  Terror 
(GWOT)  but  has  the  design  characteristics  needed  for  irregular  warfare  in  general.  In  her  paper,  Wong  discussed 
statistical  analysis  and  robustness  issues  with  respect  to  the  game.  She  also  discussed  data  mining  and  ontologies 
as  analysis  tools. 

Discussion  Topics: 

General  discussion  produced  the  following  points: 

•  Phrasing  is  wrong,  it  is  influence 
o  Deterrence  is  one  subset 
o  Others 

■  Theater  security  cooperation 

■  Dissuasion 

■  Containment 


•  Deterrence  Operations  JOC  definition  of  deterrence  assumes  rational  choice  economic  models 

o  Behavioral  aspect  needs  to  be  brought  in 

•  Don’t  necessarily  know  who  attacked  us 

o  But  can  target  terrorist  financiers,  logisticians 
o  Need  to  broaden  the  problem  space 

•  Success  only  measured  by  lack  of  behavior? 

o  Doesn’t  make  sense:  trying  to  prove  a  null 

o  Indications  of  “success”  exist  but  can  you  link  a  deterrent  action  to  an  indication? 
o  Can  we  model  deterrence  failure  instead? 

•  Decision  makers  comfortable  with  partial  solutions 

o  Small  and  focused  models  (not  mega  models) 
o  “Magic”  models  that  incorporate  everything  may  give  poor  answers 

•  Needs  /  approaches  discussion 

o  Focus  groups  suggested  as  an  approach  to  examine  deterrence  issues 
o  Need  cognitive  models  (of  who  you  are  trying  to  deter) 

o  Does  work  in  social  sciences  on  deterring  criminal  activity  apply  to  other  actors? 
o  Historical  case  studies  (e.g.,  studies  of  pre-WWI  telegrams) 

•  Other  issues: 

o  Including  influence  gives  other  metrics 
o  Deterrence  is  just  one  part  of  influence 
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o  Cyber  deterrence 

o  Challenge  of  linking  US  actions  to  outcomes 
•  Who  is  the  adversary?  (Need  to  focus  problem) 

o  To  advance  the  discussion,  you  have  to  get  concrete:  specific  examples 
o  Decompose  goals  to  create  a  process,  then  have  something  more  actionable  to  watch 

The  working  group  then  split  into  five  subgroups  to  discuss  how  different  actors  might  require  different 
approaches  to  deterrence  (the  “Isolated  small  groups”  possibility  was  not  discussed). 


X  (Actor) 

Y (Deterring  Entity) 

Z  (action  being  deterred) 

Individuals  (hackers) 

U.S.  government 

Hacking 

Small  groups  (Symbi on ese 
Liberation  Army) 

U.S.  government  (law 
enforcement) 

Violent  acts 

Isolated  small  groups  (British 
med  students) 

Organized  crime 
(Colombian  drug  cartels) 

U.S.  and  Colombian 
government 

Drug  trafficking 

State  (Saudi  Arabia) 

U.S.  government 

Building  schools  that  teach 
extremist  ideology 

Non-state  actors  (al 

Qaeda) 

U.S.  government 

Attacks  against  U.S.  targets 
(9/11  and  now) 

Figure  7.  Subgroups  of  Deterrence  WG 


Individuals: 

•  Example:  hacker 

•  Motives 

Build  reputation 
Personal  financial  gain 
Direct  damage 
Retaliation 


o 

o 

o 

o 

Issues: 

o 

o 

o 

o 

o 

Carrots: 

o 

o 

Sticks: 

o 

Tools: 

o 

o 

o 


Small  barrier  to  entry,  one  person  can  cause  significant  damage 

Identification  of  the  individual 

Attribution,  location 

Understanding  intent  and  motivation 

What  they  value 

Options  for  promoting  desired  behavior? 

Reward  for  hacking  (challenge  -  build  reputation  but  in  a  positive  way) 

Hack  back?  But  potential  escalation 

Systematically  explore  motivations  for  individuals  to  get  to  root  causes 
Profile,  understand  demographics  and  bound  solution  space 

Is  it  worth  it  to  build  expensive  tools  to  deter  an  individual?  Maybe  it  is,  because  of  the 
potential  damage 


Other  issues: 
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o  Cyber  mechanism  -  used  for  trafficking  and  other  destabilizing  issues 
o  Damage  from  propaganda 
o  Application  of  contagion  framework 
o  Cyber  bribery? 
o  Nigerian  fraud  rings 

Small  groups: 

•  Symbionese  Liberation  Army 

o  Violent  actions:  kidnapping,  hank  robbery,  agitation,  context  of  1960s  radicalism 
o  Motivations:  anti-war,  anti-authority 

•  Do  we  deter? 

o  Deter  next  generation  from  organizing 

•  Law  enforcement  authority 

o  How  well  financed,  observed  is  the  authority? 
o  How  aggressive  in  operations  and  in  infiltrating? 

•  Issues: 

o  Group  repurposing  -  deterrence  fails 

•  Why  do  groups  repurpose?  What  are  the  mechanisms? 
o  How  to  raise  flags  about  isolationist  behavior 

•  Can  we  detect  it? 

•  Can  we  model  it? 

o  Surveillance  -  does  it  deter?  (let’s  watch  UK  experience) 

•  Generalizations: 

o  Law  enforcement  approach  generalizes 
o  Model  functions  (e.g,  police  arrest,  surveillance,  patrol,  etc) 

•  Hypothesis 

o  Law  enforcement’s  effective  presence  must  have  been  factor  in  decline  of  these  groups  in 
the  US 

o  Decline  of  relative  gains;  prosperity,  social  cohesiveness,  inter-ethnic  relations  in  US 
o  Good  state  suppresses 

o  Small  group  violence  has  declined  with  the  rise  of  the  state 

•  Foreign  groups 

o  Deter,  neutralized,  kill 

o  Classifying:  part  of  larger  group?  Tools  will  likely  vary  depending  on  answer. 

Organized  crime: 

•  Example: 

o  Colombian  drug  traffickers  -  loose  coalition  of  opportunistic  groups 
o  Who  is  deterring:  US  and  Colombian  government 

•  Who  is  doing  the  smuggling? 

o  People  with  long  history,  experience,  education  in  violence  who  weren’t  offered  an  effective 
way  to  repatriate  into  society 

o  How  do  you  send  an  appetizing  message  to  these  types  of  people? 

•  Analytic  difficulties: 

o  Unintended  consequences  (rerouting  drug  flow) 

■  Nth  order  effects 
o  Adaptive  organizations 

o  Governments  try  only  part  of  the  solution  set 

■  Need  to  include  “host”  country  elements  in  tools 

o  How  to  create  a  niche  in  a  fully  formed  society  for  disaffected  members 

■  Modeling  equivalent  positions  (stature/pay/authority) 
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•  Issues: 

o  Groups  may  be  multi-functional? 

■  Drug  gangs  also  street  gangs  who  control  territory 

■  Smuggling  drugs  may  just  be  providing  $  for  other  activities 
o  Model  process  from  security  criminal  organization? 

■  Modeling  transition  of  group  purpose 
o  Corruption 

■  Do  we  apply  U.S.  view  or  their  view?  Is  their  an  “acceptable”/  “expected”  level  of 
corruption? 

■  Everyone  is  corrupt  except  our  group  -  helping  own  group  is  not  corruption  (may 
even  be  killed  otherwise) 

o  Modeling  or  data  collection  problem 

o  What  are  the  situations  when  arrests  don’t  become  convictions? 


State: 

•  Example:  Saudi  Arabia 

o  Deter  Saudi  government  from  building  schools  with  radical  ideology 
o  Context:  Wahabbism,  want  to  maintain  relations  with  the  Saudi  government 

•  Analytic  issues: 

o  Don’t  want  to  look  like  you’re  against  Islam 
o  How  fragile  is  the  Saudi  government? 
o  Variables:  perceptions  of  domestic  audience 

•  Generalizeable  issues: 

o  Overt  vs  covert  deterrence  actions 

■  Can’t  look  like  the  US  made  a  government  do  something 
o  Lack  of  intelligence  (lack  of  data) 

o  US  biases 
o  Nth  order  implications 
o  Influence,  not  deterrence 

o  Model  is  of  context,  not  just  target,  including  outside  perceptions 
o  Small  models  versus  capturing  the  context  (tendency  to  scale  up  model  to  try  to  capture 
context) 

o  Scale  about  the  seriousness  of  the  threat  when  deterring  states  (Saudi  textbooks  vs.  North 
Korean  nuclear  weapons,  Venezuelan  oil  output) 

■  How  should  models  incorporate  implications  of  deterrence  failure?  (think  allocation 
of  resources  to  n  deterrence  challenges  -  which  ones  get  resources?) 

o  Model  the  entire  state 

■  Oil:  U.S.  moving  sources  to  African  oil 

■  Osama  bin  Laden  is  a  Wahabbi 

■  Saudis  also  building  mosques  like  crazy  -  need  some  kind  of  antidote  to  rival  or  be 
an  alternative  (Saudis  really  opposed  to  other  kinds  of  groups) 

o  Likely  need  to  include  more  than  one  state  in  the  tools 

Non-state  actors: 

•  Example:  US  deterring  al  Qaeda  before  9/1 1  and  now 

•  Issues: 

o  Any  government  will  be  at  a  disadvantage  when  going  against  a  group  like  al  Qaeda  b/c  it  is 
a  set  of  nodes  that  do  anything  they  want 

o  Network  does  ideology,  financing,  but  don’t  have  rigid  command  and  punishment  structure 
o  Any  group  within  the  network  is  agile  -  can  have  their  own  targeting  plan,  able  to  adapt  to 
local  conditions 
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•  Recommendations 

o  If  can’t  deter,  can  try  to  contain  (containment  is  another  type  of  influence) 

■  Need  model  or  tools  to  describe  containment  strategies  and  try  to  see  why  some 
work  or  fail 

o  Can  try  to  accelerate  demise  of  al  Qaeda  by  encouraging  ffactionalization 

■  Model  of  fractionalization  in  non-state  actors 

■  What  types  of  exogenous  factors  increase  ffactionalization?  Also  a  weakness  of 
decentralized  networks 

o  Copycat  effect 

■  How  do  we  model  it? 
o  Sterilize  environment 

■  Try  to  change  operational  environment  to  change  ability  of  al  Qaeda  to  spread;  and 
support  successful  local  efforts  financially 

■  Need  tools  to  model  spread 
o  Modeling  deflection 

■  Changing  potential  terrorist  target  characteristics  to  make  them  less  vulnerable  (and 
understand  how  that  causes  a  shift  in  likely  targets) 

o  Ideological  battle,  US  should  be  quiet 

o  Raise  profile  of  SMEs  and  on-the-ground  people:  improve  their  access  to  decision  makers 
(nobody  was  listening  before  9/11  even  though  bin  Laden’s  activities  were  clear  to  Sudanese 
experts) 

•  Issues: 

o  Deflection:  US  and  Israel  deflected  embassy  attacks  (analogy  from  criminal  literature  shows 
that  police  action  deflects  criminal  activity  to  other  areas) 

•  US  deterring 

o  Too  many  franchises  under  the  umbrella,  extreme  decentralization  makes  them  impossible  to 
deter 

•  Recommendations 

o  If  can’t  deter,  can  try  to  contain  (containment  is  another  type  of  influence) 

■  Need  model  or  tools  to  describe  containment  strategies  and  try  to  see  why  some 
work  or  fail 

o  Best  to  let  regional  actors  in  the  Middle  East  -  they  have  better  ways  to  influence 

■  Watch  what  is  working  in  Saudi  Arabia,  etc. 

■  Means  models  must  be  broad  (numerous  countries) 

The  group  returned  to  a  general  discussion  with  the  following  points: 

•  Whole  range  of  possibilities  for  influencing  makes  sense 

o  Deflect,  deter,  influence,  attract,  etc. 

•  Unintended  consequences 

•  Building  tools  -  the  next  workshop? 

o  What  kinds  of  tools  are  being  built? 
o  Do  we  know  how  to  build  models? 

Expressed  needs: 

•  Particular  questions 

o  Deterring  hacking 
o  Deterring  violent  acts  by  small  groups 
o  Deterring  drug  trafficking 
o  Deterring  states  from  unwanted  social  actions 
o  Deterring  non-state  actors 

•  Supporting  research  needs 
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o  Cognitive  models  (of  who  you  are  trying  to  deter) 

o  Does  work  in  social  sciences  on  deterring  criminal  activity  apply  to  other  actors? 
o  Historical  case  studies  (e.g.,  studies  of  pre-WWI  telegrams) 
o  Understanding  intent,  motivations  and  things  valued  by  hackers 
o  Does  surveillance  deter  small  groups 
o  Influencing  organized  crime  groups 
o  Understanding  and  defining  corruption 
o  Defining  economic  alternatives  for  organized  crime  members 
o  Understanding  containment  strategies  for  non-state  actors 
o  Understanding  ffactionalization  of  non-state  actors 
o  Understanding  fragility  of  governments 
o  Influencing  states 
Model  architecture  needs 

o  Small  and  focused  models  (not  mega  models) 


2.3.2  COIN 


Membership: 

•  Jack  Goldstone,  George  Mason  University  (chair) 

•  Brett  Pierson,  J8  WAD  (co-chair) 

•  Chuck  Barry 

•  David  Carment 

•  Conrad  Crane 

•  Richard  Cunningham 

•  Jennifer  Edwards 

•  Roger  Hillson 

•  Larry  Kuznar 

•  Lauren  Lee 

•  Michael  Manno 

•  Dennis  McBride 

•  Anne  McGee 

•  Lisa  Moya 

•  Richard  Pei 

•  Victoria  Romero 

•  A1  Sciarretta 

•  Stephen  M.  Shellman 

•  Barry  Silverman 

•  A1  Sweetser 

•  Mike  Vlahos 

•  Paul  Wise 

•  Paul  Works 

Presentations: 

CDR  Brett  Pierson,  has  served  as  the  Commanding  Officer  of  VFA-147  and  is  now  at  The  Joint  Staff/J8 
Warfighting  Analysis  Division.  He  presented  “The  Hairball  that  Stabilized  Iraq:  Modeling  FM  3-24.” 

In  December  2005,  an  Army-Marine  Corps  writing  team  began  revising  the  existing  Army  field  manual  on 
Counterinsurgency.  Guided  by  LTG  David  Petraeus  and  LTG  James  Mattis,  the  group  strove  to  balance  the 
lessons  of  the  past  with  contemporary  insights  and  future  projections.  Early  in  the  process  the  team  wrestled  with 
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the  overall  approach  to  be  taken.  One  option  was  an  enemy-centric  approach  that  would  emphasize  the 
elimination  of  opposing  combatants  as  the  key  to  long  term  success.  Realizing  that  such  activities  were  an 
important  element  of  any  COIN  strategy,  the  team  decided  that  persistent  success  could  only  come  from  a 
population-centric  approach  which  aimed  to  gain  the  support  of  the  people  for  the  counterinsurgent.  From  that 
decision  flowed  a  broad  comprehensive  doctrine  emphasizing  a  whole  range  of  activities  beyond  just  security, 
involving  a  set  of  intertwined  and  supporting  Logical  Lines  of  Operation. 

CDR  Pierson  then  explained  the  lines  that  show  the  doctrine  and  comprise  the  model  of  FM  3-24,  shown  in 
Figure  8. 


Dr.  Jack  A.  Goldstone,  Virginia  E.  and  John  T.  Hazel  Jr.  Professor  of  Public  Policy  and  Eminent  Scholar,  George 
Mason  School  of  Public  Policy,  presented  “Modeling  Macro-systemic  Change  for  Counter-insurgency.” 

The  systems  model  presented  by  CDR  Pierson  gives  a  good  sense  of  the  complexity  of  the  relationships  involved 
in  suppressing  counter-insurgencies.  One  must  pay  attention  to  the  dynamic  relationships  driving  the  insurgents, 
affecting  the  broader  populace,  determining  the  effectiveness  of  the  military  and  police,  and  underlying  the 
stability  or  instability  of  the  local  and  national  governments.  This  is  a  multi-player  game  with  changing 
conditions,  and  thus  one  of  hardest  situations  to  model  with  simple  rules  and  equilibrium  outcomes.  According  to 
Goldstone,  reconstruction  rests  on  four  major  pillars:  security  provision,  economic  reconstruction  (jobs  and 
incomes),  political  reconstruction  (self-sustaining  and  administratively  effective  government),  and  social  service 
delivery  (health,  education,  sanitation,  energy,  transport).  However,  Goldstone  finds  the  usefulness  of  the  model 
is  limited  to  its  schematic  representation  and  the  support  it  gives  to  decision  makers  in  building  a  set  of  categories 
around  which  to  frame  their  own  mental  models,  rather  than  any  use  in  running  the  model  on  a  computer. 

Goldstone  described  an  eight-fold  matrix,  in  which  each  pillar  is  tracked  by  a  cluster  of  measurements  of  which 
some  aim  to  track  changes  in  security,  economic  growth,  services,  and  government,  while  others  aim  to  track  the 
fairness  of  those  changes  through  distributional  measures.  However,  global  metrics  are  meaningless  in  specific 
cases.  He  said  that  each  country  case  requires  its  own  set  of  metrics.  He  did  suggest  the  following  metrics: 

(1)  Security:  Body-counts,  but  not  where  more  bodies  of  insurgents  is  good.  Rather,  body- 

counts  caused  by  insurgent  and  government  action  are  both  bad. 
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(2)  Governance:  The  critical  overall  element  of  governance  is  not  participation,  or  equality, 
or  democracy  -  it  is  trust.  If  one  can  get  truthful  answers  to  a  single  question:  “How  much  do 
you  trust  the  government  to  protect  you  and  act  responsibly?”  one  can  measure  governance  much 
more  effectively  than  by  looking  at  constitutions  or  officials’  actions. 

(3)  Economics:  Economists  have  taught  us  that  the  key  to  economic  stability  is  not  current 
data,  but  expectations. 

(4)  Social  Services:  The  key  to  such  services  is  not  simply  to  provide  them,  but  to  make  sure 
that  they  can  be  sustained  by  the  local  or  national  government  once  foreign  support  is  withdrawn. 

Discussion  Topics: 

•  Additional  information  needed  for  COIN 

o  Better  definition  of  COIN 

■  Include  situation  in  which  country  has  weak  or  no  government 

■  Coordinate  definition  with  other  related  areas:  counterterrorism,  SSTRO 

•  Terminology  problem:  Too  many  terms  that  overlap;  have  semantic  inflation 
o  Better  understanding  of  need  for  COIN  tools  above/below  brigades 

■  How  do  we  delineate  which  tools  are  for  the  right  people 

•  E.g.,  BDE  Cdr  needs  a  community  model  to  see  possible  issues  at  his  level 

■  Solicit  feedback  from  small  units  about  information  needs 

•  FM  3-24  model  is  a  higher  level  model,  need  simpler  tools  for  small  units 

o  If  we  allowed  people  to  look  domestically  at  conflict,  would  we  get  farther  along  to  build  the 
methodologies  and  the  underlying  functions  of  the  models 

■  How  would  it  be  different  for  non-domestic  modeling? 

•  FM  3-24  Model 

o  Uses  system  dynamics  to  model  COIN 
o  Need  to  address  agent-based  models  in  the  model 
o  Has  not  been  validated  with  real-world  data 
o  What  type  of  research  is  needed  to  make  this  a  practical  tool?: 

■  Not  an  enemy-centric  model  (e.g.,  A1  Qaeda  did  a  lot  of  things  wrong) 

■  Program  is  hard-wired  from  US  viewpoint  -  if  you  don’t  get  with  the  U.S.  program  then 
you’re  an  enemy 

o  Model  has  three  terms  that  need  FAR  better  definition  and  recognizable  measures 

■  Understanding  and  knowledge  of  social  structures 

■  Appropriate  mix  of  effort  and  use  of  force 

■  PSYOPS  effectiveness 

o  Need  for  understanding  local  legitimacy 

o  Need  better  understanding  of  why/how  people  move  in  groups  not  as  individuals  from  one  state 
(e.g.,  neutral)  to  another  (e.g.,  pro-government) 
o  Need  to  understand  the  impact  of  criminal  element 
o  Need  to  include  inter-Agency  involvement  (the  DI  E  pieces) 
o  Need  to  design  models  to  be  able  to  handle  more  than  one  insurgency  at  a  time 

■  Non-state  actors;  religious  sects;  criminal 

•  May  attack  each  with  a  different  line  (military,  diplomatic,  economic) 

o  FM  3-24  model  is  focused  on  reduction  of  violence  -  need  other  assessment  of  other  insurgent 
actions  and  outcomes 

o  Need  to  “unpack”  what  is  included  in  “Support  Insurgency” 

•  Other  considerations 

o  Consider  using  psychologists  from  industry 
o  Need  to  be  more  open  about  what  we  are  doing  -  pos.  image 
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■  Proceed  with  caution  to  prevent  anti-DoD  view 

o  Need  closer  relationships  with  Non-DoD  organizations  and  NGOs 

Expressed  needs: 

•  Particular  questions  -  none 

•  Supporting  research  needs 

o  Need  for  understanding  local  legitimacy 

o  Need  better  understanding  of  why/how  people  move  in  groups  not  as  individuals  from  one  state 
(e.g.,  neutral)  to  another  (e.g.,  pro-government) 
o  Need  to  understand  the  impact  of  criminal  element 
o  Need  to  include  inter- Agency  involvement  (the  DI  E  pieces) 
o  Ability  to  model  “trust” 
o  Ability  to  add  “govemability”  in  the  analysis 
o  MOPs  and  MOEs  for  assessing  progress 
o  Know  how  people  will  react  to  an  action 

o  Understand/know  about  social  structures:  leaders/networks/norms 
o  Data  on  local  social  structures 
o  Environmental  data  that  influences  EISCB 
o  Include  medical  data  (e.g.,  HIV  positive) 

•  Model  architecture  needs 

o  Need  to  design  models  to  be  able  to  handle  more  than  one  insurgency  at  a  time 

■  Non-state  actors;  religious  sects;  criminal 

•  May  attack  each  with  a  different  line  (military,  diplomatic,  economic) 
o  Models  that  are 

■  Tailorable 

■  Updated  within  a  resource  repository 

■  Complex  and  adaptive  while  running 
o  Bounds  for  complex,  adaptive  models 

o  Hybrid  models  with  fine  grain  locations  for  addressing  particular  issues 
o  A  systems  architecture  so  different  models  can  inform  each  other 
o  Integration  of  data,  models,  and  systems  (how  do  the  4  clubs  interact?) 

■  Design  to  allow  output  of  one  model  (e.g.,  agent-based)  to  feed  the  system  dynamics  of 
another  system  component 

■  Need  mapping  of  data  across  models 

•  Include  hand-shake  across  modeling  modalities 

■  Supporting  data  needs  some  agreed  on  and  clearly  defined  objectives 

■  Need  cross-disciplinary  understanding  across  modelers  and  social  scientists 

■  Need  an  integrated  DIME  model 

o  Need  bits  and  pieces  of  models  that  can  used  by  all  and  then  build  from  there 

■  Framework  for  tailoring  the  architecture 

o  Data  standards,  data  storage,  data  mining,  data  extraction/discovery  tools 
o  VV&A 

■  How  do  we  VV&A  tools  in  a  world  of  competitive  analysis 

•  For  example,  joint  output  that  opposes  a  Service  may  be  discredited  by  a  pro- 
Service  SME 

•  How  to  get  something  useful  to  a  3 -Star  that  will  withstand  scrutiny 

■  Need  for  better  evidence-based  assessment 

■  Need  access  to  social  science  people  who  WANT  to  help 

■  How  often  do  we  update  the  model  and  VV&A  it? 
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2.3.3  CT 


Membership: 

•  Guermantes  Lailari,  USAF  (chair) 

•  Dipak  Gupta,  San  Diego  State  University  (co-chair) 

•  Eli  Berman 

•  Deanna  Caputo 

•  Claudio  Cioffi-Revilla 

•  Jesse  Citizen 

•  Gary  Citrenbaum 

•  Lisa  Costa 

•  Jennifer  Edwards 

•  Brian  Efird 

•  Teresa  Embury 

•  Ari  Greenberg 

•  Margaret  Hayes 

•  JC  Hertz 

•  Max  Lorenzo 

•  Jimmie  McEver 

•  Charlene  Milliken 

•  Will  Moore 

•  MattNickens 

•  Maureen  O’ Mara 

•  Michael  Ottenberg 

•  Peter  Pham 

•  Jerrold  Post 

•  Kevin  Roney 

•  Bob  Sheldon 

•  Ted  Woodcock 

Presentations: 

Lt  Col  Guermantes  Lailari,  self-described  career  Air  Force  officer,  presented  “The  Challenges  of  Modeling  and 
Simulating  Terrorism  and  Counterterrorism.” 

According  to  Lailari,  the  key  point  in  M&S  is  the  validation  process.  With  this  in  mind,  Lailari’ s  presentation 
portrayed  the  various  assumptions  about  M&S,  what  it  can  bring  to  the  fight  and  what  it  cannot  (capabilities  and 
limitations).  Lailari  went  on  to  discuss  various  technical  aspects  of  terrorism  -  its  means  and  targets. 

Dr.  Dipak  K.  Gupta  is  a  Distinguished  Professor  in  the  Department  of  Political  Science  at  San  Diego  State 
University  and  the  Fred  J.  Hansen  Professor  of  Peace  Studies.  He  is  also  the  Director  of  the  International  Security 
and  Conflict  Resolution,  a  multidisciplinary  undergraduate  program.  His  areas  of  expertise  include  ethnic 
conflict,  collective  action,  public  policy  analysis,  and  quantitative  methods.  Dr.  Gupta  presented  “Modeling  the 
Dynamics  of  Terrorist  Movements:  A  Macro  Interactive  Perspective.” 

According  to  Gupta,  the  level  of  violence  from  terrorism  and  insurgency  is  the  outcome  of  a  dynamic  interaction 
between  a  dissident  group  —  which  strategically  use  violence  —  and  the  target  government,  which  offers  to  engage 
it  with  force.  The  past  quarter  of  a  century  saw  an  ever-increasing  cascade  of  empirical  studies  showing  the  link 
between  macro  economic  and  social  structural  variables  to  the  level  of  conflict  in  a  society.  Despite  popular 
belief  that  poverty,  income  inequality,  and  lack  political  freedom  cause  terrorism,  the  econometric  studies  have 
come  up  largely  empty  handed.  The  reason  for  this  apparent  lack  of  correlation  rests  with  the  fact  that  the  while 
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the  structural  variables  offer  the  necessary  condition  for  political  violence,  the  sufficient  cause  rests  with  the 
ability  of  a  political  entrepreneur  to  take  the  aspects  of  frustration  and  frame  the  grievances  in  terms  of  a  matter  of 
collective  identity,  complete  with  a  plan  of  action.  The  introduction  of  an  external  catalytic  agent  robs  empirical 
models  of  predictive  capabilities.  However,  while  prediction  may  be  problematic,  a  careful  evaluation  of  the 
factors  that  contributes  to  the  terrorism  increasing  forces  (TIF)  and  terrorism  attenuating  forces  (TAF)  opens  up 
new  avenues  for  its  management. 

Discussion  Topics: 

•  Multi-disciplinary  approach 

o  Encourage  the  marketplace  of  ideas 

•  A  =  Ultimate  Outcome: 

o  Decreasing  violence  and  increasing  governance 

•  B  =  Interim  Outcome:  theory  dependent  (social  sciences) 

•  C  =  Research  Question  (RFP) 

•  Organize  CT  goals  by  ultimate  and  interim  outcomes  (C  B  A) 

o  For  example:  interim:  moderating  violence,  encouraging  defection 

•  Establishing  causal  relationships  to  achieve  the  ultimate  outcome 

•  CT  Buckets 

o  Terrorism 
o  Counterterrorism 
o  State 
o  Non-State 
o  Other  States 

•  Overlap 

Expressed  needs: 

•  Particular  questions 

o  How  to  decrease  violence  and  increase  governance 
o  How  to  moderate  violence,  encouraging  defection 

•  Supporting  research  needs  -  none 

•  Model  architecture  needs  -  none 

2.3,4  SSTRO 

Membership: 

•  Paul  West,  USMA  (chair) 

•  Richard  Deckro,  AFIT  (co-chair) 

•  Cherie  Beck 

•  Jonathan  Beris 

•  Jeff  Burkhalter 

•  Alok  Chaturvedi 

•  Doug  Clark 

•  Dave  Davis 

•  Tom  Ferlemen 

•  LeeEllen  Friedland 

•  Rebecca  Goolsby 

•  John  Hummel 

•  Leroy  "Jack"  Jackson 

•  John  Lawson 

•  Danielle  Miller 
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•  Brice  Nicholson 

•  Jay  Persons 

•  John  Salerno 

•  Barry  Silverman 

•  Dan  Snyder 

•  Eric  Whittington 

•  Victor  Wiley 

Presentations: 

Dr.  Paul  D.  West,  Assistant  Professor  in  the  Department  of  Systems  Engineering,  United  States  Military 
Academy,  where  he  teaches  systems  engineering,  decision  analysis,  engineering  economics,  and  modeling  and 
simulation.  West  presented  “Human,  Social,  and  Cultural  Behavior  Modeling  for  Stability,  Security,  Transition, 
and  Reconstruction  Operations.” 

West  presented  concepts  and  issues  in  HCSB  modeling  for  SSTRO,  examined  several  tools  in  development  or 
use,  and  outlined  a  new  approach,  called  Dynamic  Natural  Attribute  (DNA)  modeling,  for  generating  unique 
computer-generated  entities. 

Dr.  Richard  F.  Deckro,  Department  of  Operational  Sciences,  Air  Force  Institute  of  Technology,  presented  “Issues 
in  Stability,  Security,  Transition  and  Reconstruction  Operations.” 

Deckro  provided  an  overview  of  some  of  the  issues  present  in  conducting  stability,  security,  transition  and 
reconstruction  operations  in  today’s  operational  environment.  He  reviewed  several  key  directives,  reports,  and 
definitions  and  discussed  a  number  of  potential  problem  areas  as  areas  for  future  research. 

Discussion  Topics: 

The  discussion  was  oriented  on  the  question  of  what  should  models  do  in  2018. 

•  Assure  that  we  understand  the  micro-climate  of  an  area  before  entry,  the  linkages  between  elements  of 
SSTRO  and  how  they  feed  into  one  another  (inputs/outputs) 

o  Maintaining  robustness  of  the  model  in  a  dynamic  environment, 
o  Avoid  fighting  not  only  the  last  war,  but  the  last  campaign? 

•  Prioritize  SSTRO  to  serve  total  U.S.  interests  (e.g.,  DoD,  DOS)  as  well  as  host  nation,  coalition,  and 
broader  international  community 

•  Balance  transition  to  indigenous  groups  (clans,  tribes,  regional,  or  national),  maintaining  integrity  of  host 
nation  decisions  while  supporting  U.S.  interests. 

o  What  if  they  democratically  vote  in  a  theocracy? 
o  How  do  we  not  impose  our  will  on  a  free  and  democratic  society? 

The  group  identified  several  limiters  to  SSTRO  modeling 

•  Ability  to  obtain,  organize,  and  access  data  (e.g.,  metadata) 

•  Policy  issues  for  open  data  access 

•  Fidelity  requirements  (these  are  unknown) 

•  Available  resources 

•  Fines  of  authority 

•  Clash  of  cultures  between  models  (e.g.,  ethics  by  discipline;  focus) 

Out  of  the  discussion,  several  key  insights  emerged: 

•  Need  new  look  at  the  problem  from  new  eyes 

•  Need  cross-disciplinary  theory  of  SSTRO 

o  If  you  don’t  know  where  you  are  going,  it  does  not  matter  how  you  get  there 
o  How  to  prioritize  SSTRO 
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•  Model  requirements 

o  Model  must  consider  NGO,  Coalition,  and  host  nation  factors  at  multiple  scales 
o  Model  must  be  sufficiently  flexible  and  robust  to  respond  to  unknown  future  national  security 
interests 

o  Data  availability;  open  shared  repository  as  appropriate.  Meta  data  protocol;  require  in  BAA? 
o  Need  appropriate  sensitivity  analysis 

•  SSTRO  “ownership”  is  bigger  than  DoD  -  It  must  span  DoD,  DOS,  DHS,  providing  for 

o  Common  definitions 
o  Database  architecture  management 
o  Unity  of  effort 

o  Establishing  a  Community  of  Interest,  communicating  electronically 

Expressed  needs: 

•  Particular  questions 

o  Understanding  the  operation 
o  Prioritizing  resources  to  perform  the  operation 
o  Balancing  flow  through  the  SSTR  process 

o  Assure  that  we  understand  the  micro-climate  of  an  area  before  entry,  the  linkages  between 
elements  of  SSTRO  and  how  they  feed  into  one  another  (Inputs/outputs) 
o  Balance  transition  to  indigenous  groups  (clans,  tribes,  regional,  or  national),  maintaining  integrity 
of  host  nation  decisions  while  supporting  US  interests 
o  Prioritize  SSTRO  to  serve  total  US  interests  (e.g.,  DoD,  DOS)  as  well  as  host  nation,  coalition, 
and  broader  international  community 

•  Supporting  research  needs 

o  Need  cross-disciplinary  theory  of  SSTRO 
o  Accepted  measures  of  effectiveness  and  performance 
o  Clearly  defined  initial  state 

o  Clearly  defined  end  state  -  What  does  success  look  like? 
o  Thorough  understanding  of  the  underlying  theories 

■  Social 

■  Physical 

•  Model  architecture  needs 

o  Representation  of  a  wide  spectrum  of  cultures  and  institutions  (formal  and  informal) 
o  Representation  at  multiple  scales  for  Blue,  Red,  Green 

■  Temporal 

■  Spatial 

■  Demographic 

o  Representation  of  behaviors  during  conflict,  disaster,  etc 
o  Flexibility  in  representing  different  types  of  operations  supporting  U.S.  interests 
o  Criteria  for  thresholds  to  determine  when  transitions  occur  between  S,  S,  T,  and  R 
o  Criteria  for  distinguishing  friendly  locals  from  hostile 
o  Representation  of  inter-organizational  dynamics 
o  Representation  of  situational  awareness 

o  Representation  of  dynamic,  competing  views/needs/priorities  by  all  groups 
o  Analytical  capability  to  evaluate  balance/  tradeoff  options  and  conduct  sensitivity  analysis 
o  Collaborative  planning  capability  that  spans  neighborhood  to  national  (possibly  multi-national) 
level 

o  Architecture  the  supports  PMESII  M&S  interoperability  and  reuse 
o  “Service  bus”  for  data  repositories 

■  Provides  management  and  error  checking 

o  Dynamic  Data  Driven  Application  Systems  with  real-time  forecasting 
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o  Ability  to  obtain,  organize,  and  access  data  (e.g.,  metadata) 
o  Policy  issues  for  open  data  access 
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3.  FUTURE  HSCB  TASKS 


The  workshop  participants  were  asked  to  suggest  future  HSCB  tasks.  These  suggestions  have  been  organized  into 
five  groups: 

•  Metadata,  Metamodels,  and  Support  Structures, 

•  Collect  Data, 

•  Validate  Existing  Models, 

•  Build  HSCB  Models,  and 

•  Education. 

3.1  METADATA,  METAMODELS,  AND  SUPPORT  STRUCTURES 

3.1.1  Identify  Data  and  Its  Sources 

HSCB  models  will  have  wide  variations  in  their  data  requirements.  Some  of  the  data  will  need  to  be  current  and 
detailed  data  for  a  specific  situation.  However,  some  data  requirements  can  be  met  by  historical  data  or  from 
analysis  of  historical  data  (e.g.,  trends).  Numerous  data  sources  exist;  however,  some  are  derivative,  with  the 
potential  for  introduced  errors,  and  some  are  derivative  with  corrections  added.  Model  users  will  need 
identification  and  descriptions  (metadata)  for  the  various  sources.  In  addition  to  the  descriptions  of  the  data  items, 
descriptions  and  explanations  of  the  range  of  methodologies  that  are  used  to  gather  data  (from  true  experimental 
through  naturalistic  qualitative  observation)  will  aid  in  informing  users  about  the  choices  to  be  made. 

Data  on  human  behavior  related  to  non-traditional  challenges  is  a  DoD-wide  problem.  Although  DoD  is  making 
significant  investments  that  often  yield  data  that  would  be  useful  to  other  users  within  DoD,  it  can  be  difficult  to 
find.  Having  the  three  major  DoD  investment  areas  in  social  science  (HSCB,  Minerva,  and  Human  Terrain)  adopt 
a  common  metadata  standard  for  data  generated  through  their  projects  would  benefit  many  users. 

Data  Definition  Project: 

•  Start  with  PITF  data  gathering  project  and  extant  data  list 

•  Consider  whether  HSCB  modeling  requires  a  larger  data  set 

•  Define  metadata  requirements 

•  Parcel  out  conceptual  islands  for  validation  of  concepts-measurements 

•  Gather  and  analyze  data 

o  Commission  searches  through  social  sciences  professional  societies 
o  Hold  a  conference  to  refine  metadata  and  data  V&V 

o  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging  purposes 

3.1.2  Identify  the  Theories  of  the  Social  Sciences 

Several  people  bemoaned  the  lack  of  a  set  of  codified  theorems,  generalizations,  and  “rules”  in  social  science. 
Others  noted  that  there  are  theories,  but  few  of  them  have  been  validated  empirically.  In  fact,  in  anthropology 
alone  there  are  uncounted  theories.  They  are  based  on  observed  behavior,  but  few  enjoy  unanimous  acceptance. 
There  is  also  the  issue  of  fragmentation  within  the  field,  which  means  that  nowadays  anthropology  and  probably 
the  other  social  sciences  are  producing  fewer  generalists  and  more  people  with  very  narrow  areas  of  expertise. 
This  works  against  the  production  of  broad  theories. 

Theory  Definition  Project: 

•  Start  with  extant  theory  list 
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•  Define  metadata  requirements 

•  Gather  and  analyze  theories  to  models 

o  Commission  searches  through  social  sciences  professional  societies 
o  Hold  a  conference  to  refine  metadata  and  theory  V&V 

o  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging  purposes 

Theory  Validation  Project: 

If  a  list  of  validated  theories  or  generalizations  is  seen  to  be  necessary,  then  scholars  should  be  paid  to  gather  the 
studies  that  validate  or  refute  theories.  This  would  be  a  long-range  effort  that  might  not  yield  as  much  fruit  as 
could  be  hoped.  But  a  start  could  be  made. 

3.1.3  Identify  the  Methods  of  OR  and  the  Social  Sciences  Relevant  to  HSCB  Modeling 

HSCB  models  will  inevitably  be  composed  of  both  social  science  theories  and  OR  methods.  Some  OR  methods 
will  be  clearly  applicable  to  HSCB  modeling,  others  may  be  clearly  inapplicable;  however,  many  methods  may  be 
useful  in  some  context,  but  will  require  some  thought  to  identify  the  proper  contexts. 

Methods  Definition  Project: 

•  Start  with  the  Encyclopedia  of  Operations  Research  and  Management  Science 

•  Define  metadata  requirements 

•  Gather  and  analyze  methods 

o  Commission  searches  through  INFORMS  and  MORS 
o  Hold  a  conference  to  refine  metadata  and  method  V&V 

o  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging  purposes 

3.1.4  Identify  the  HSCB  Models 

Numerous  models  claim  to  be  useful  in  the  DIME/PMESII  domain;  however,  the  extant  lists  of  such  models  do 
not  contain  enough  information  to  determine  in  which  part  of  the  domain  they  might  be  useful.  This  effort  might 
be  coordinated  with  the  VV&A  area,  below. 

Models  Definition  Project: 

•  Start  with  the  extant  list  of  DIME/PMESII  models  (Hartley,  NRL,  JFCOM,  OSD  PA&E) 

•  Define  metadata  requirements  (start  with  DIME/PMESII  Model  VV&A  Tool  and  above  sources) 

•  Commission  a  multi-disciplinary  group  to  perform  rough  V&V  for  metadata  tagging  purposes 

3.1.5  Identify  the  MOMs  for  HSCB  Modeling 

Proper  Measures  of  Merit  (MOMs)  are  critical  for  useful  HSCB  modeling.  Clearly  using  a  bad  MOM  can  lead  to 
incorrect  decisions.  However,  it  may  be  impossible  or  very  difficult  to  model  some  MOMs  directly.  It  will  take 
some  thought  to  determine  whether  it  is  better  to  model  a  particular  MOM  imperfectly  or  model  several  MOMs 
fairly  well  and  infer  the  desired  MOM. 

MOMs  Definition  Project: 

•  Hold  a  MOMs  Workshop  (after  defining  data,  theories,  methods,  and  models) 

o  Outcomes  vs  outputs  vs  inputs  measures  [counting  #  of  flyers  distributed  is  not  same  as 
measuring  the  impact  of  the  information  operation] 
o  Desired  MOMs  vs  availability  of  data  vs  model  conversion  of  data  to  MOMs 
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3.1.6  Create  a  Common  Lexicon 


A  major  requirement  for  HSCB  modeling  is  a  vocabulary  to  promote  a  common  understanding  of  social  science 
and  its  disciplines  in  order  to  drill  down  to  determine  what  potential  expertise,  data,  tools,  theories,  methods,  and 
models  may  be  required  for  the  development  of  particular  tools.  There  are  two  candidate  methodologies  for 
organizing  and  maintaining  a  common  vocabulary  or  lexicon:  the  Data  Documentation  Initiative  (DDI)  and  the 
Unified  Medical  Language  System  (UMLS). 

•  The  DDI  3.0  is  an  international  XML  metadata  standard  for  social  science  data  and  datasets.  Members  in 
the  DDI  Alliance  include  major  U.S.  research  universities  (Berkeley,  Harvard,  Stanford,  Yale,  Cornell, 
MIT,  Princeton,  Wisconsin,  UCLA,  and  others)  as  well  as  several  European  social  science  data  archives. 
DDI  development  was  supported  by  a  grant  by  the  National  Science  Foundation  (NSF). 

•  The  alternative  is  to  use  the  National  Fibrary  of  Medicine’s  UMFS,  with  its  Metathesaurus,  Semantic 
Network,  and  SPECIAFIST  Fexicon,  as  a  basis  for  the  effort.  UMFS  was  developed  because  of  the 
critical  need  for  accurate  communication  of  medical  concepts. 

Common  Lexicon  Project: 

•  Investigate  candidate  methodologies  for  organizing  and  maintaining  a  common  vocabulary  or  lexicon. 

•  Adoption  one  of  the  methodologies. 

•  Begin  the  process  of  creating  the  common  lexicon  using  the  methodology. 

•  Encourage  adoption  by  all  HSCB  projects  and  by  Project  Minerva  and  Human  Terrain  projects. 

3.1.7  Identify  the  Proper  Data  Library  Concept 

Data  on  human  behavior  related  to  non-traditional  challenges  is  a  DoD-wide  problem.  Although  DoD  is  making 
significant  investments  that  often  yield  data  that  would  be  useful  to  other  users  within  DoD,  it  can  be  difficult  to 
find.  A  data  and  metadata  registry  for  HSCB  data  would  also  permit  researchers  in  FFRDCs,  academia,  and 
industry  easier  access  to  data  that  may  prove  important  to  the  further  development  of  basic  and  applied  science  to 
support  HSCB  modeling. 

Data  Library  Project: 

•  Hold  a  Data  Library  Concept  Workshop  (after  defining  data,  theories,  methods,  and  models) 

o  Consider  the  potential  problems  of  a  Data  Library 

o  Consider  the  options:  central  library,  virtual  library  (distributed),  government  vs  academic  vs 
joint 

o  Consider  the  legal  issues 

o  Consider  the  data  maintenance,  creation,  VV&A,  tagging,  and  funding  issues 

3.1.8  Government  Office 

Create  a  governmental  activity  that  is  responsible  for  and  has  authority  over  HSCB  modeling.  This  activity  might 
be  a  completely  new  organization  or  it  could  be  created  within  an  existing  organization  by  adding  responsibilities, 
authority  and  funding.  Possible  existing  organizations  are  the  M&S  CO  within  DDR&E  and  the  Simulation  & 
Analysis  Center  (SAC)  or  Joint  Data  Support  (JDS)  within  OSD  PA&E.  All  of  these  suffer  from  being  part  of 
DoD,  which  might  cause  difficulties  within  other  parts  of  the  government. 

However,  the  presence  of  scientific  experts  who  could  provide  expertise  users  developing  models  and  tools  that 
involve  social  science  knowledge  would  be  useful.  There  is  a  critical  gap  in  social  science  expertise  in  the 
development  of  DoD  models  and  tools.  Many  efforts  suffer  from  substantial  inclusion  of  credible  theory  or  data, 
and  many  program  managers  overseeing  model  development  are  not  familiar  enough  with  the  social  sciences  to 
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even  know  where  to  look  for  expertise.  Further,  interaction  with  scientific  experts  should  increase  the  social 
science  “literacy”  of  DoD  personnel  and  contractors  using  HSCB  tools  over  time.  It  may  also  be  beneficial  to 
have  designated,  interdisciplinary  social  science  teams  for  specific  issue  areas  of  interest  to  DoD,  such  as  stability, 
security,  reconstruction,  and  transition  (SSTR). 

If  many  of  the  scientific  positions  were  IPAs,  this  would  continually  refresh  the  scientific  base  and  currency  on 
the  latest  research.  In  addition  to  benefiting  individual  users,  an  office  of  interdisciplinary  experts  consulting 
frequently  with  many  parts  of  DoD  would  have  the  visibility  into  activities  to  better  recommend  other  HSCB 
investments  that  address  common  problems.  There  is  also  the  benefit  that  such  an  office  could  judge  the  quality 
of  products  being  contracted  out  through  the  HSCB  process. 

Government  Office  Project: 

•  Establish  an  SSTR  research  program  (near  term:  examine  MPICE  and  the  ISSM) 

•  Establish  other  specialty  areas  that  are  important  to  several  communities 

•  Prioritize  gaps  in  basic  research  and  primary  data  collection  that  would  benefit  a  wide  number  of  DoD 
users 

3.1.9  Create  a  Code  of  Best  Practice  for  HSCB  Modeling 
Contact  Names:  Dr.  Stuart  Starr,  Dr.  Dean  Hartley,  and  Mr.  A1  Sciarretta 

Commonality:  The  Acquisition,  Training,  Analysis  and  Experimentation  communities,  within  and  among 
Services,  and  possibly  others,  all  use  M&S,  yet  many  users  lack  knowledge  of  best  practices  in  key  areas.  This 
lack  of  knowledge  is  particularly  notable  in  the  areas  of:  data  models  and  standards;  creation  and  orchestration  of 
live,  virtual,  and  constructive  simulation;  reuse  of  M&S;  verification,  validation,  and  accreditation  (VV&A); 
selection  of  Measures  of  Merit;  selection  and  application  of  experimental  designs  to  guide  the  use  of  M&S;  the 
tradeoffs  associated  with  alternative  architectural  approaches  (e.g.,  DIS,  HLA,  TENA);  and  the  social  skills 
needed  to  identify  properly  the  problem  to  be  addressed,  identify  the  significant  stakeholders,  and  obtain  buy-in 
for  the  project  plan.  These  shortfalls  are  most  seriously  felt  in  project  lead  positions  where  each  of  these  areas 
must  be  addressed,  yet  the  time  and  effort  of  each  must  be  balanced  against  a  budget  and  the  overall  success  of  the 
project.  In  the  absence  of  that  knowledge,  there  is  widespread  failure  to  make  the  best  use  of  M&S  in  its  creation 
and  application.  Further,  the  absence  of  that  knowledge  by  decision  makers  permits  occasional  abuses. 

User/Customer:  All  DoD  Components,  including  representatives  from  the  aforementioned  communities,  and  the 
defense  industrial  base. 

Deliverables:  The  deliverables  of  this  proposed  effort  consist  of  documentation  and  dissemination  of  the  COBP 
for  M&S.  The  deliverables  are  described  as  unitary  items;  however,  they  are  segmented  as  follows:  common 
elements,  analysis  specific  elements,  acquisition  specific  elements,  training  specific  elements,  and 
experimentation  elements. 

•  A  concise  set  of  guidelines  to  convey  M&S  best  practices  to  the  users  of  M&S  and  decision  makers  who 
receive  the  results  of  the  application  of  M&S. 

o  A  brochure,  summarizing  major  insights,  which  can  be  disseminated  to  professional  associations 
(e.g.,  MORS,  INFORMS,  ITEA). 

o  A  check  list  for  a  decision  maker,  to  inform  them  of  community  best  practices, 
o  An  executive  summary  of  the  COBP  for  publication  in  Phalanx  and  other  professional 
publications. 

•  A  COPB  manual.  This  manual  will  be  a  richly  annotated  document  on  best  practices  that  can  be  used  in 
curricula  at  universities. 

•  An  intensive  campaign  to  disseminate  widely  the  guidelines  to  the  targeted  audiences.  The  material  from 
the  book  will  be  used  to  delivery  tutorials  at  key  professional  associations  (e.g.,  MORS,  INFORMS, 
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ITEA,  NDIA)  and  professional  military  education  opportunities  (e.g.,  CAPSTONE).  It  will  be  provided  to 
teachers  at  key  schools  that  teach  M&S,  leveraging  MORS’  Education  Colloquium 

Functions:  As  identified  in  community  business  plans,  the  Department  needs  a  holistic,  well-informed 
assessment  of  the  issues  and  choices  in  this  area,  to  include  literally  best  practices,  where  they  can  be  identified, 
and  “better”  practices,  where  distinctions  can  be  made  between  good  and  bad  practices  but  the  truly  best  practices 
cannot  be  determined.  This  project  will  assess  the  issues  and  choices  and  identify  the  category  to  which  each  of 
the  various  “best”  practices  belong. 

Constraints:  To  solicit  innovative  ideas  and  build  ownership  for  the  proposed  path  ahead,  the  project  must  be 
done  in  an  open,  objective  and  unclassified  manner.  As  they  become  available,  draft  deliverables  will  be 
distributed  for  review  and  comment  by  the  various  user  communities.  An  interim  report(s)  will  be  completed, 
followed  by  refinement,  final  invitation  for  comments,  and  production  of  the  final  deliverables. 

Work  Description:  The  associated  tasks  for  the  deliverables  are  provided  below.  The  tasks  are  described  as 
unitary  items;  however,  they  are  segmented  as  follows:  common  elements,  analysis  specific  elements,  acquisition 
specific  elements,  training  specific  elements,  and  experimentation  elements. 

•  Task  1.  Review  existing  codes  of  best  practice. 

•  Task  2.  Based  on  that  review,  derive  key  lessons  and  select  an  appropriate  framework  for  the  COBP. 

One  possible  framework  is  to  structure  the  COBP  around  the  M&S  lifecycle  (e.g.,  initial  data  activities; 
model  creation;  model  development;  M&S  application;  data,  information  use;  treatment  of  residual  issues 
(e.g.,  model  reuse)). 

•  Task  3.  Once  the  framework  is  selected,  systematically  identify  best  practices  associated  with  the  use  of 
M&S  to  support  basic  functions.  Although  the  main  focus  of  the  project  will  be  on  best  practices  for 
M&S  that  are  relevant  for  all  communities,  the  study  will  also  identify  community  unique  best  practices 
in  analysis,  acquisition,  planning,  testing,  training,  experimentation  areas  to  the  extent  that  funding 
permits. 

•  Task  4.  Codify  best  practices  based  on  key  community  products,  discussions  with  subject  matter  experts, 
and  the  experience  of  the  study  team. 

•  Task  5.  Submit  these  preliminary  products  to  a  Greybeard  Panel  for  review. 

•  Task  6.  Based  on  feedback  from  the  Greybeards,  refine  the  enumerated  best  practices  and  identify 
residual  issues  that  require  additional  research. 

•  Task  7.  Document  the  results  of  these  analyses  in  multiple  products. 

•  Task  8.  Conduct  a  campaign  to  disseminate  widely  its  products  and  insights. 

Success  Criteria:  Success  will  be  judged  by  support  for,  and  commitment  to  the  COBP  for  M&S  by  the  using 
communities. 

•  A  key  metric  will  be  the  number  of  users  of  M&S  who  are  exposed  to  the  COBP  through  the  deliverables 
cited  above. 

•  A  second  metric  will  be  the  number  of  decision  makers  who  are  exposed  to  the  COBP  through  the 
deliverables  cited  above. 

•  A  third  metric  will  be  the  availability  of  the  documentation  as  living  documents  on  at  least  one  web  site. 

Recently,  key  decision  makers  have  mandated  that  users  employ  COBPs  to  guide  their  activities.  For  example,  the 
ASD  (Nil)  and  the  UK  MoD  have  recently  mandated  that  C2  assessments  must  adhere  to  the  NATO  COBP  for 
C2  Assessment.  Ultimately,  we  would  seek  to  have  key  decision  makers  mandate  that  analysts  employing  M&S 
adhere  to  the  COBP  for  M&S  for  Analysts. 

3.2  COLLECT  DATA 

One  of  the  most  important  areas  that  the  HCSB  effort  can  contribute  to  is  data  collection;  however,  not  just  any 
data  collection.  In  particular,  we  need  more  information  on  insurgency  and  COIN  operations.  Brett  Pierson  has 
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designed  a  systems  dynamics  model  that  articulates  the  causal  processes  outlined  in  the  FM  3-24  COIN  manual. 
While  this  model  highlights  important  causal  connections,  it  has  not  been  evaluated  with  real  data.  One  of  the 
most  important  ways  forward  is  to  call  for  the  collection  of  data  important  to  evaluating  COIN  operations. 

Within  this  model  there  are  five  major  embedded  variables:  essential  services,  governance,  economic 
development,  perceived  security,  and  host  nation  security  forces.  Each  of  these  concepts  affects  and  is  affected  by 
other  important  variables.  Some  of  these  include  psychological  operations  effectiveness,  appropriate  strategic 
emphasis,  impact  of  illegitimate  actions,  and  satisfaction  with  essential  services.  We  probably  have  relatively  few 
good  ways  to  operationalize  these  concepts  and  the  others  that  populate  the  model.  In  order  to  know  if  this  model 
tells  us  important  things  about  COIN  operations  and  the  right  levels  of  force,  economic  activity,  governance,  etc. 
to  implement,  we  need  to  generate  quality  data  for  each  of  these  concepts  and  use  the  data  to  test  the  model  using 
various  quantitative  methods  (agent  based  models,  statistics,  etc.).  More  importantly,  we  should  generate  new 
methods  and  leverage  old  ones  in  terms  of  the  data  collection  efforts.  With  advances  in  computing  technologies 
we  should  explore  new  ways  of  collecting  information  using  automated  and  computerized  techniques.  We  need 
data  in  near-real  time  in  order  to  feed  this  model  and  others  developed  to  truly  understand  what  works  and  what 
does  not  in  COIN  operations. 

Collect  Historical  Data  on  COIN  Project: 

•  Historical  data  on  insurgencies  and  counter-insurgencies. 

•  Create  new  technologies  to  generate  new  data  on  insurgency  and  counter-insurgency 

3.3  VALIDATE  EXISTING  MODELS 

Verification,  Validation  &  Accreditation  (VV&A)  as  a  unified  activity  is  required  by  the  Department  of  Defense 
(DoD);  however,  it  is  either  rarely  done  or  rarely  reported.  Further,  where  it  is  reported,  the  details  are  often 
omitted. 

The  recommendations  from  the  VV&A  Working  Group  are  based  on  a  statement  of  value  for  exposure  of  the 
details  of  VV&A,  not  only  for  models,  but  also  for  data,  theories,  and  methods.  For  years  (going  back  at  least  to 
1974),  various  elements  within  the  DoD  has  been  cataloging  models.  These  catalogs  characterize  the  models 
using  a  fairly  short  list  of  tags,  such  as  stochastic  or  deterministic  and  closed  form  or  human  in  the  loop.  More 
recently  several  organizations  (including  my  own)  have  cataloged  DIME/PMESII  models.  These  catalogs  have 
never  achieved  their  full  potential  in  utility  because  their  tags  don’t  go  deep  enough.  They  don’t  allow  a  user  to 
find  the  model  or  models  that  best  address  that  user’s  needs. 

The  VV&A  Working  Group  created  the  beginning  of  a  list  of  characteristics  that,  if  known,  would  serve  to 
support  the  identification  of  models  (data,  theories,  and  methods)  that  do  address  a  user’s  needs.  The  Working 
Group  further  identified  the  V&V  process  as  the  proper  process  for  determining  the  values  of  these  characteristics 
for  a  particular  item.  However,  to  be  useful,  the  characteristics  need  to  be  standardized,  the  V&V  must  actually 
be  performed,  and  the  results  must  be  “published.” 

A  project  to  perform  sufficient  V&V  activities  on  a  selected  set  of  models  to  determine  the  values  of  the 
characteristics  will  require  funding.  Funding  will  also  be  required  for  the  model  developers  to  support  their 
required  participation  in  these  activities.  It  is  estimated  that  two  to  three  weeks  of  concentrated  work  would  be 
sufficient  for  each  model.  The  DIME/PMESII  Model  VV&A  Tool  would  be  used  to  record  the  results. 

The  results  of  this  project  are  not  sufficient  for  the  larger  community  needs;  however,  they  are  sufficient  to 
determine  the  value  of  the  approach.  If  the  value  is  demonstrated  to  be  large  enough,  DoD  can  require  that  all 
HSCB  models  undergo  this  process. 

Validate  Several  Existing  Models  Project: 

•  Fund  a  small  group  (approximately  6  people)  to  organize,  run,  and  report  on  the  project. 

•  Organize  characterization  workshops,  with  paid  participants. 
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o  Workshop  1  to  define  the  necessary  characteristics  of  models, 
o  Workshop  2  to  define  the  necessary  characteristics  of  data, 
o  Additional  workshops  for  theories  and  methods  might  be  needed. 

•  Validate  several  models  and  tools 

o  Establish  validation  standards  for  HSCB  products  and  the  limits  of  acceptable  use 
o  Validate  products  of  potential  interest  to  a  wide  number  of  users,  such  as  COMPOEX 

■  COMPOEX 

■  MPICE 

■  ISSM 

■  SEAS 

■  FAST  Toolbox 

o  Report  the  results  and  supply  a  database  that  supports  user  search  and  retrieval  of  models  that 
match  user  requirements 

3.4  BUILD  HSCB  MODELS 

Three  different  HSCB  model  building  projects  are  described  below. 

3.4.1  Build  A  Model  Using  an  Existing  Argonne  National  Laboratory  Framework 

Situations  that  need  HSCB  models  generally  require  multi-agency  and  inter-governmental  responses.  The  Areas- 
of-Operation  (AOs)  where  the  activities  occur  include  a  landscape  of  physical  (e.g.,  natural  environment, 
infrastructure,  military  force  structures,  etc.),  human,  social,  and  cultural  components  that  must  be  constructed 
and  analyzed  in  an  integrated,  synergistic  fashion  against  the  particular  goals  and  objectives  of  a  given  mission  or 
operation. 

Figure  9  provides  an  example  of  a  representative  landscape  that  has  been  constructed  in  support  of  studies 
undertaken  by  the  Argonne  National  Laboratory.  In  the  example  shown,  the  problem  involved  analyses  of  the 
impact  of  regulations  on  an  electrical  infrastructure.  In  this  example,  the  landscape  consisted  of  a  physical  layer 
of  the  physical  infrastructure  of  the  electrical  system  being  studied;  three  business  layers  that  incoiporated  human 
and  social  aspects;  and  a  regulatory  layer  that  considered  cultural  issues  (i.e.,  state  level  regulations.)  As  shown 
in  the  example,  the  individual  layers  are  linked  together,  which  means  that  an  activity  taken  within  one  layer  can 
have  impacts  that  are  felt  by  other  layers. 

The  example  shown  represents  an  application  for  a  specific  problem  domain.  However,  the  concepts 
demonstrated  in  this  example  can  be  extended  to  HSCB  modeling.  The  differences  would  lie  at  the  conceptual 
level  in  the  types  of  layers  that  would  be  required  for  a  given  problem  and  at  the  implementation  level  in  the 
entities  and  their  behaviors  required  for  the  problem,  as  well  as  the  data  sources  required  to  describe  and  populate 
them. 

In  responding  to  HSCB  needs,  there  would  not  be  a  “one  size  fits  all”  solution.  Instead,  a  framework  environment 
that  permits  the  addition  of  easily  integrated  context-appropriate  tools  provides  a  robust  solution  to  problem  of 
constructing  and  analyzing  human,  social,  cultural,  and  behavioral  landscapes.  Argonne  has  considerable 
experience  in  constructing  and  using  human,  social,  cultural,  and  behavioral  applications  from  a  generalized 
framework  perspective  and  these  Argonne  developed  frameworks  could  be  used  in  applications  involving  HSCB 
landscapes. 
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Figure  9.  An  Example  of  Landscape  Layers  Used  to  Analyze  the  Impact  of  Regulations  on  an  Electrical 

Infrastructure 


Figure  10  provides  a  high  level  summary  of  applications  developed  by  Argonne  that  involve  HSCB  interactions. 
These  applications  were  developed  using  two  Argonne  simulation  development  environments  -  the  Dynamic 
Information  Architecture  System  (DIAS)  and  Repast.  DIAS  is  a  subject  domain  independent  flexible,  extensible, 
object-oriented  framework  for  developing  and  maintaining  complex  simulations.  DIAS  is  patented,  but  is  licensed 
royalty  free  to  U.S.  Government  agencies  and  their  contractors.  Repast  is  a  free,  open  source  agent-based 
modeling  and  simulation  library  of  tools  designed  to  support  the  development  of  extremely  flexible  models  of 
social  agents. 


Application 

Description 

CASCADE-CD 

A  complex  adaptive  systems  application  for  use  in  developing  and 
analyzing  counter-drug  interdiction  strategies. 

CASCADE-FA 

A  complex  adaptive  systems  application  for  use  in  developing  and 
analyzing  Joint  force  campaign  operations,  with  emphasis  on  time 
sensitive  targeting. 

CASCADE-ES 

A  complex  adaptive  systems  application  for  the  evaluation  of  stability 
and  sustainability  of  societies  under  stress. 

ENKIMDU 

A  simulation  of  ancient  Mesopotamia  for  studying  long-term  human- 
environmental  interactions. 

Mae  Phosop 

An  extension  of  the  ENKIMDU  simulation  system  to  modern  Thailand 
to  obtain  agroeconomic  insights  into  the  dynamics  of  rural  economies. 

EMCAS 

A  complex  adaptive  system  model  of  electrical  energy  markets. 

Figure  10.  High  Level  Summary  of  Argonne  Applications  Involving  Human,  Social,  Cultural,  and  Behavioral 

Interactions 


3.4.2  Build  Small  Models  Using  Thick  Ethnographies 

There  is  an  apparent  difference  of  opinion  about  -  or  at  least  varying  degrees  of  comfort  with  -  the  depth  of  social 
and  cultural  knowledge  that  should  be  built  into  the  models.  On  one  extreme  is  the  type  of  thick  description  that 
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Partnow  and  Hartley  their  plenary  paper.  Close  on  its  heels  is  the  detailed  inclusion  of  a  number  of  psychological 
and  sociological  variables  in  Barry  Silverman’s  presentation.  At  the  other  end  are  simplified  models  that  make 
use  of  readily  available  economic  and  sociological  statistics  such  as  economic  indicators  and  infant  mortality. 
Somewhere  in  the  middle  lie  the  sociology-based  studies  of  social  networks.  David  Siegel  suggested  that  his 
models,  which  could  be  placed  toward  the  simplified  end,  but  not  at  the  extreme,  indicate  that  one  can  eliminate 
thick  description  and  still  have  valid  and  useful  models.  Still,  his  models  are  fairly  limited  in  what  they  can 
predict.  While  there  is  room  for  all  levels  of  social  science  engagement,  the  more  inclusive  end  of  the  spectrum  is 
the  missing  piece  of  the  puzzle  now.  It  is  missing,  despite  its  potential  to  produce  description,  identify  causation, 
and  predict  behavior  based  on  cultural  values,  practices,  and  beliefs. 

Two  questions  inspired  by  Silverman’s  paper  are,  “Is  it  really  possible  and  feasible  to  build  models  (other  than 
pictorial  representations)  that  include  thick  description?  And  is  the  endeavor  so  time  consuming  that  it  is  neither 
cost  effective  nor  useful?”  Silverman’s  models  were  very  involved,  and  yet  (based  only  on  his  presentation)  they 
were  still  incomplete  -  they  did  not  include  cultural  information  of  the  sort  advocated  in  the  Partnow/Hartley 
paper.  Perhaps  the  question  should  be  reframed:  “What  sorts  of  problems  might  HSCB  be  useful  for?”  The 
question  should  be  broken  into  small  units  so  that  depth  can  be  achieved.  For  instance,  huge  questions,  such  as, 
“Will  this  government  topple  under  various  circumstances?”  might  not  be  appropriate  to  a  thick  description-type 
model.  A  better  problem  for  such  a  method  might  be,  “How  should  US  forces  introduce  elements  of 
infrastructure  to  a  particular  locale?”  Anthropology  can  contribute  a  great  deal  to  the  micro-level  of  modeling. 

The  involvement  of  more  social  scientists  requires  that  social  scientists  become  re-educated.  Those  who  already 
employ  statistical  methods  are  prime  candidates  -  however,  the  “purer”  cultural  anthropologists  should  not  be 
discounted.  Rather,  actual  working  workshops  in  which  ethnographers  partner  with  modelers  to  build  a  few 
prototypes  are  needed.  This  will  serve  to  train  both  sets  of  practitioners  and  will  also  be  an  opportunity  to  test  the 
kinds  of  problems  that  are  best  served  by  HSCB  models. 

Build  Small  HSCB  Models  Project: 

•  Hold  small,  longer  duration  workshops 
o  Ethnographers 
o  Modelers 

o  Build  a  prototype  model 

3.4.3  Build  a  Country  Monitoring  System 

The  figure  below  illustrates  the  architecture  of  a  global  country  monitoring  system.  The  local  experts  in  each 
country  and  the  regional  experts  make  the  data  entries  in  their  own  offices.  The  local  and  regional  users  view  the 
results  in  their  own  offices.  The  data  are  processed  at  a  central  location,  using  the  Interim  Semi-static  Stability 
Model  (ISSM),  a  proven  tool  for  making  inferences  regarding  the  stability  situation  at  the  country  or  sub-country 
level,  given  input  of  a  relatively  small  amount  of  observable  data.  The  ISSM  makes  and  tracks  the  inferences 
over  time.  It  also  provides  for  the  addition  of  custom  logic  to  create  and  track  non-standard  inferences. 
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This  system  minimizes  the  human  costs  of  data  entry,  while  maximizing  the  validity  of  the  data  through  use  of 
those  with  the  best  knowledge.  The  system  provides  incentives  for  the  people  who  must  feed  it  in  the  form  of 
feedback  of  information  about  the  country  for  which  they  are  responsible  and  information  about  its  neighbors. 

As  indicated  in  the  figure,  the  system  can  be  extended  by  adding  automated  feeds  of  certain  information  from  the 
electronic  news  sources.  This  extension  requires  the  use  of  computer  parsing  software,  which  is  available  from 
several  sources. 

Build  Country  Monitoring  System  Project: 

•  Construct  architecture 

•  Modify  applications  software  (ISSM) 

•  Create  automated  input  subsystem 

•  Distribute  and  run  test  case 

3.4,4  Build  an  Open  Source  Model  Framework 

The  concept  of  this  proposal  is  that  we  build  the  Impact  Analysis  simulation  as  an  open  model,  with  contributions 
from  the  entire  Social  Sciences  community.  The  simulation  architecture  in  the  specification  document  describes 
the  objects  to  be  modeled  and  their  general  relationships;  however,  the  specific  relationships,  political,  financial, 
and  social,  should  be  determined  by  experts  in  the  appropriate  fields.  Moreover,  the  structure  of  the  model  will 
call  for  potentially  as  yet  poorly  defined  relationships,  identifying  needed  research. 

This  simulation  would  be  a  “medium-weight”  model.  That  is,  it  should  have  a  smaller  footprint  and  lesser  input 
requirements  than  SEAS,  but  larger  than  the  ISSM.  It  should  be  constructed  in  a  similar  manner  to  open-source 
Linux,  with  a  central  core  of  support  designers  and  programmers,  and  be  open  to  contributions  from  anyone. 

We  would  start  with  architecture  workshops,  in  which  the  decisions  about  the  types  of  interactions,  relationships, 
and  modeling  styles  (e.g.,  stochastic  elements)  would  be  made  -  but  not  decisions  on  the  theories  and  algorithms 
that  should  be  used  to  implement  the  relationships.  At  this  point,  we  could  build  the  model’s  framework  -  that  is 


54 


the  simulation  engine,  output  map  displays,  input  and  output  data  stores,  and  so  forth,  and  implement  the  result  in 
code.  We  would  have  to  be  careful  to  make  the  attributes  for  the  objects  to  support  additions,  as  we  could  not  be 
certain  what  attributes  would  be  needed  to  support  instantiated  theories. 

At  this  point  the  application  programming  interface  (API)  would  be  exposed  to  the  community  at  large.  Social 
Scientists  could  propose  theories  to  instantiate  selected  relationships  or  interactions.  They  could  team  with 
modelers  to  implement  the  theories  in  code  that  could  be  added  to  the  model.  They  would  have  the  option  of 
calling  for  additional  attributes  to  be  added  to  support  the  code.  They  would  also  be  required  to  provide  data 
descriptions  and,  if  possible,  current  sources  for  the  data  needed  for  their  theory.  Naturally,  they  would  also  be 
required  to  supply  complete  documentation.  Multiple  theories  could  be  implemented  to  model  a  single 
relationship,  where  no  single  theory  is  known  to  be  correct. 

The  core  team  would  provide  configuration  management  and  verification  &  validation  (V&V). 

This  project  should  be  denoted  as  a  long-term  R&D  project  to  provide  a  test-bed  for  HSCB  theories  and  a  source 
for  prioritizing  the  needs  for  basic  research  in  HSCB  theories.  As  competing  theories  are  implemented,  they 
could  be  tested  against  each  other  and  against  the  real  world.  Also,  as  parts  of  the  model  are  implemented,  it 
would  become  clear  which  parts  are  being  left  vacant,  indicating  areas  of  need. 

3.5  EDUCATION 

Educate  Modelers:  Based  on  this  two-day  workshop,  it  appears  to  me  that  the  “modeling  community”  is  primed 
to  accept  social  scientists,  at  varying  levels  of  involvement.  I  believe  their  acceptance  will  gradually  grow  with 
more  information  and  more  workshops  of  this  type.  I  was  struck  by  the  fact  that  my  very  basic  anthropological 
outline  was  seen  as  a  new  approach  by  many  of  the  engineers  in  the  room.  I  imagine  there  are  many  other 
modelers  who  will  need  to  be  introduced  to  the  social  sciences  in  a  similar,  simplified,  manner. 

Educate  Social  Scientists:  Because  HSCB  modeling  is  not  well  known  in  anthropological  circles,  an  effort  has 
to  be  made  to  invite  scholars  in.  This  can  be  accomplished  through  papers  at  professional  conferences,  as  well  as 
invitations  to  the  type  of  workshop  suggested  above.  Those  people  already  involved  should  use  their  established 
social  and  professional  networks  to  recruit  others,  with  continuing  efforts  to  extend  those  networks. 

•  The  involvement  of  more  social  scientists  requires  that  we  become  re-educated  as  well,  for  the  reasons 
noted  in  a  paragraph  above.  Those  who  already  employ  statistical  methods  are  prime  candidates  -  but  I 
don’t  think  the  “purer”  cultural  anthropologists  should  be  discounted.  Rather,  I  would  recommend  actual 
working  workshops  in  which  ethnographers  partner  with  modelers  to  build  a  few  prototypes.  This  will 
serve  to  train  both  sets  of  practitioners  and  will  also  be  an  opportunity  to  test  the  kinds  of  problems  that 
are  best  served  by  HSCB  models. 

•  Getting  more  social  scientists  involved  seems  to  be  an  issue.  Based  on  the  participation  in  this  workshop, 
it  is  apparent  that  some  sociologists,  economists,  psychologists,  and  anthropologists  are  already  involved, 
but  far  more  (particularly  anthropologists)  should  be  part  of  the  endeavor.  Factors  currently  limiting 
anthropologists’  participation  include: 

o  This  is  not  a  career  track  that  is  well  known  within  the  profession. 

o  Unless  employment  is  assured  at  some  level,  it  is  not  likely  to  become  a  standard  career  track  for 
anthropologists 

o  There  is  a  mismatch  between  the  study  and  practice  of  anthropology  and  HSCB.  One  area  of 
dissonance  is  the  use  of  statistical  methods.  Although  some  subfields  of  anthropology  regularly 
use  statistics  (archaeology,  economic  anthropology,  ecological  anthropology),  most  cultural 
anthropologists  have  not  used  statistics  since  their  graduate  school  days,  and  do  not  find  it 
appropriate  to  ethnographic  description.  In  fact,  statistical  data  is  generally  considered  less 
valuable  than  other  types,  showing  effects  rather  than  underlying  causes. 
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Build  Small  HSCB  Models  Project: 

•  Hold  small,  longer  duration  workshops 
o  Ethnographers 
o  Modelers 

o  Build  a  prototype  model 
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4.  SYNTHESIS 


The  purpose  of  a  workshop  is  to  elicit  something  from  the  participants.  This  process  has  been  referred  to  as 
attempting  to  “herd  a  bunch  of  cats.”  In  general,  the  brighter  the  participants,  the  more  valuable  are  the  results; 
however,  it  is  also  generally  true  that  brighter  participants  are  harder  to  herd.  The  purpose  of  a  synthesis  group  is 
to  identify,  capture  and  organize  the  results  of  a  workshop.  Each  working  group  produces  some  organization  of 
its  results;  however,  there  may  be  valuable  utterances  that  are  expressed  in  a  working  group  that  are  not  well 
connected  to  that  group’s  purpose.  The  embedded  members  of  the  synthesis  group  attempt  to  capture  these. 
Further,  there  may  be  themes  that  are  common  across  the  groups  that  should  be  connected.  Finally,  the  synthesis 
group  looks  for  opportunities  to  find  synergies  from  the  working  group  results,  synthesizing  results  that  would  not 
be  apparent  from  a  bare  reading  of  the  individual  group’s  results. 

Figure  12  shows  the  organization  of  the  results  of  the  workshop  that  directly  addressed  its  purpose:  characterizing 
the  capabilities  needed  to  perform  effective  HSCB  modeling  in  support  of  operational  users  and  senior  decision 
makers.  The  drivers  are  the  questions  that  operational  users  and  senior  decision  makers  want  answered.  From 
these  flow  the  implied  needs  in  both  the  social  sciences  and  the  modeling  domains.  From  these  flow  indirect 
needs.  The  practice  of  Operations  Research  is  shown  linking  these  needs  together. 


4.1  PARTICULAR  QUESTIONS 

Observation:  Many  questions  can  be  categorized  as  Predict  something,  Support  an  activity,  Balance  competing 
activities,  Prioritize  among  competing  options,  or  Understand  something 
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•  Predict 

o  General  prediction  &  forecasts 
o  refugee  flow  during  political  crises:  inadequate 

o  the  flow  of  contagions  through  the  population-density  map  of  a  society:  good 
o  effects  on  societies  institutions  of  long-term  epidemic  PTSD:  inadequate 
o  when  cultural  change  will  be  regarded  as  authentic 
o  best  COA 

•  Support 

o  Theater  Security  Cooperation  Plans  (TSCPs) 
o  situational  awareness 

o  Identifying  enduring,  reoccurring  questions  that  arise  in  military  operations 

•  Balance 

o  flow  through  the  SSTR  process 

o  transition  to  indigenous  groups  (clans,  tribes,  regional,  or  national),  maintaining  integrity  of  host 
nation  decisions  while  supporting  US  interests 

•  Prioritize 

o  resources  to  perform  the  operation 

o  Given  limited  $$,  what  part  of  DIME  should  we  invested  in? 

o  SSTROs  to  serve  total  US  interests  (e.g.,  DoD,  DOS)  as  well  as  host  nation,  coalition,  and 
broader  international  community 

•  Understand 

o  exploration  (branches/sequels)  and  discovery  &  exploration  of  black  swans 
o  the  operation 

o  If  the  military  does  a  task  (for  example,  give  water  in  an  SSTR  task),  what  are  the  implications 
from  a  societal  point  of  view? 
o  What  to  look  for  in  a  village  you  are  going  through? 
o  Understand  failed  states 

■  History 

■  Economy 

■  Political  contents 

■  Ecological  and  demographic  factors 

■  Natural  resources,  both  for  internal  use  and  as  exports 

■  Military  and  paramilitary  forces 

■  Culture  and  linguistic  situation 

■  Racio-ethnic  dimensions 

■  Religious  factors 

■  Gender  and  social  order 

■  External  influences  on  all  of  the  above,  e.g.,  globalization 
o  Legitimacy 

o  Motive/intent,  Goals  (explicit,  implicit,  public,  private) 
o  Leadership  profiles 
o  Family  structure/dynamics 
o  Peer  influences 
o  Why  people  become  insurgents? 

■  Why  are  people  X  becoming  Insurgent  Y  in  location  Z? 

■  Which  of  these  people  are  becoming  insurgents? 

■  How  do  I  “reverse  this”?  How  long?  What  will  it  take? 

■  How  are  the  red  team  going  to  respond  to  our  actions?  Who  is  the  red  team? 
o  What  are  the  consequences  including  unintended  of  our  actions? 
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o  the  micro-climate  of  an  area  before  entry,  the  linkages  between  elements  of  SSTRO  and  how  they 
feed  into  one  another  (Inputs/outputs) 
o  How  to  decrease  violence  and  increase  governance 
o  How  to  moderate  violence,  encouraging  defection 
o  Deterring  hacking 
o  Deterring  violent  acts  by  small  groups 
o  Deterring  drug  trafficking 
o  Deterring  states  from  unwanted  social  actions 
o  Deterring  non-state  actors 

o  What  are  the  system  drivers  /  interaction  mechanisms  of  the  results. 

4.2  SOCIAL  SCIENCE  NEEDS 

The  social  sciences  portion  of  the  HSCB  needs  is  organized  into  Definitions,  Measures  of  Merit,  Basic  Research, 
Theories,  and  Data.  The  data  needs  actually  span  both  social  sciences  and  modeling;  however,  they  are  placed  in 
this  section  for  convenience. 

4.2.1  Definitions 

Observation:  Many  HSCB-related  definitions  are  too  vague,  ambiguous  (e.g.,  “counterinsurgency”)  and  many  are 
discipline  specific  and  not  widely  understood 

•  terminology  among  the  social  sciences 

•  Develop  more  specific  definitions  to  drive  action 

•  terminology  -  common  ontology  is  too  restrictive 

•  Central  Library  with  concepts,  variables,  metadata  (Data  dictionaries,  semantic  descriptions,  ontologies) 

•  HSCB  modeling  requires  standards  and  definitions  of  common  terms  for  both  OR  and  social  science 
communities 

•  Standards  for  data  lexicon,  semantics,  and  ontology 

o  Must  include  metadata  &  HSCB  protocols 
o  Must  be  developed  within  the  real-world  context  of  the  model 

4.2.2  Measures  of  Merit  (MOMs) 

Observation:  We  do  a  poor  job  in  formulating  Measures  of  Performance  and  Measures  of  Merit  for  HSCB,  and 
developing  the  relationships  (cause  and  effect)  that  link  them 

•  We  need  to  tailor  the  MoMs  to  HSCB  problems  of  interest  and  develop  relationships  that  link  them 

•  We  need  to  display  HSCB  MoMs  to  operational  users  and  senior  decision  makers  in  a  fashion  that 
conveys  appropriate  levels  of  uncertainty  and  risk 

•  Accepted  measures  of  effectiveness  and  performance 

•  Concepts  to  outcomes  (measures  of  merit)  connections 

•  Clearly  defined  initial  state,  Clearly  defined  end  state  -  What  does  success  look  like? 

•  MOPs  and  MOEs  for  assessing  progress 

4.2.3  Basic  Research 

Observation:  Many  critical  HSCB  areas  are  not  well  understood  at  the  social  sciences  level 

•  Understand  interior  cognitive  models 

•  Knowledge,  data,  and  algorithms  that  account  for  the  effects  of  influences  (operational  activities)  on  the 
attitude/behavior  of  the  civilian  population  based  on  ethnic,  tribal,  cultural,  religious,  and  political 
considerations 
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•  local  legitimacy 

•  why/how  people  move  in  groups  not  as  individuals  from  one  state  (e.g.,  neutral)  to  another  (e.g.,  pro- 
government) 

•  impact  of  criminal  element 

•  Ability  to  model  “trust” 

•  Ability  to  add  “govemability”  in  the  analysis 

•  how  people  will  react  to  an  action 

•  social  structures:  leaders/networks/norms 

•  inter-Agency  involvement  (the  DI  E  pieces) 

•  Historical  case  studies  (e.g.,  studies  of  pre-WWI  telegrams) 

•  intent,  motivations  and  things  valued  by  hackers 

•  defining  corruption 

•  containment  strategies  for  non-state  actors 

•  fractionalization  of  non-state  actors 

•  fragility  of  governments 

4.2.4  Theories 

Observation:  In  selected  areas  of  social  science,  there  are  excessive  numbers  of  theories  (e.g.,  Barry  Silverman 
noted  several  theories  that  purport  to  characterize  “the  root  causes  of  terrorism”).  However,  there  are  key  areas 
where  needed  theories  are  lacking  or  are  too  particular  or  too  general 

•  When  multiple  theories  exist,  we  need  codes  of  best  practice  /  guidelines  on  which  theory  to  use,  when 

•  We  need  to  develop  additional  social  science  theories  to  address  issues  of  interest  for  national  security 

•  HSCB  that  drive  how  the  models  function? 

•  Need  cross-disciplinary  theory  of  SSTRO 

•  Physical  underlying  theories 

•  that  translate  civilian  attitudes  into  levels  of  cooperation  with  friendly  forces  and  result  in  corresponding 
levels  of  HUMINT  provided  by  the  civilian  population 

•  to  account  for  discrimination  between  civilian  and  adversary  actors  based  on  presented  physical  and 
behavioral  signature  (e.g.,  insurgent  in  civilian  clothing) 

•  that  account  for  how  affiliations  and  support  for  other  actors  change  based  on  the  application  of 
influences  (e.g.,  friendly  operations,  government  activities,  adversary  operations) 

•  account  for  changes  in  target  audience  attitudes  caused  by  the  application  of  PSYOPS 

•  that  reflect  adversary  HUMINT  networks  (e.g.,  attributes  of  the  HUMINT  network,  how  the  network  is 
formed,  how  the  network  adjusts  if  a  node  or  element  is  removed,  what  adversary  activities  tend  to 
facilitate  or  discourage  the  population’s  provision  of  HUMINT) 

•  to  account  for  unique  adversary  PSYOPS  techniques  and  the  effect  of  those  techniques  on  the  target 
audience 

•  to  represent  the  effects  of  CMO  on  the  attitudes  of  the  civilian  population  (or  other  target  audience) 

•  the  effects  of  essential  services  (or  lack  thereof)  on  civilian  population’s  attitudes/behaviors 

•  accounting  for  the  effect  of  governmental  corruption  on  governmental  institutions  and  on  the  civilian 
population’s  attitudes/behaviors 

•  reflecting  the  attitudes/behavior  of  actors  (e.g.  civilian  population)  based  on  the  state  of  physical 
infrastructure  used  by  the  actors 

•  accounting  for  non-homogeneous  groups  of  actors.  This  is  particularly  problematic  when  group 
members  have  overlapping  affiliations  (e.g.  a  single  actor  or  group  belongs  to  multiple  groups  -  religious 
groups,  ethnic  groups,  political  groups,  tribal  groups,  etc.) 

•  accounting  for  levels  of  civilian  support  for  and  the  provision  of  physical  or  monetary  resources  to 
adversaries 
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•  accounting  for  the  performance/effectiveness  of  actor  organizations  based  on  the  level  of  cooperation 
between  those  organizations  (e.g.  USAID  and  local  government) 

•  accounting  for  the  behavior  of  actors  (e.g.  civilian  population,  religious  leaders)  based  on  their  level  of 
support  for  the  existing  government 

•  accounting  for  the  legitimacy  of  the  existing  government  as  viewed  from  outside  the  nation  by  external 
groups  and  the  effect  of  international  legitimacy  on  government  effectiveness 

•  accounting  for  the  of  the  state  of  the  existing  legal  system  and  its  impact  on  the  attitudes/behaviors  of  the 
civilian  population 

•  how  the  civilian  attitudes/behaviors  change  as  the  economic  conditions  change 

•  the  effects  of  media  activity  on  the  attitudes/behaviors  of  actors 

•  the  effects  of  friendly  operations  on  media  themes  and  activity 

•  Cognitive  models  (of  who  you  are  trying  to  deter) 

•  Does  work  in  social  sciences  on  deterring  criminal  activity  apply  to  other  actors? 

•  Does  surveillance  deter  small  groups 

•  Influencing  organized  crime  groups 

•  Influencing  states 

•  Defining  economic  alternatives  for  organized  crime  members 

4.2.5  Data 

Observation:  HSCB  data  are  diffused,  difficult  to  find  and  access.  HSCB  data  lack  necessary  information  to 
support  analysis  (e.g.,  adequate  metadata,  indications  of  pedigree).  [Standard/original  source]  data  sets  are  never 
really  ready  for  use  -  they  require  clean  up  to  fit  the  current  need. 

•  Data,  what  exists?  What  doesn’t  exist? 

•  Data  needs  to  be  validated 

•  Central  Library  with  defined  granularity  &  validation,  needs  to  include  qualitative  data  as  well  as 
quantitative  data  Data  Integrity,  Consistency,  Maintenance,  and  Dissemination  are  required 

•  Develop  appropriate  HSCB  taxonomies,  ontologies,. . . 

•  Perform  an  assessment  of  the  desirability  of  a  Central  HSCB  Data  Repository  (issues:  classification, 
access,  open  source  data,  legal,  granularity,  qualitative  data,  maintenance,  dissemination) 

•  Implement  efforts  to  tailor  HSCB  data  to  satisfy  the  intended  purposes 

•  Permanent  repository  of  rules/relationships  like  Cyc  for  the  social  sciences 

•  Reliability  and  Validity  Tests  of  operational  qualitative  and  quantitative  variables  must  be  performed 

•  Legal  issues  regarding  data  collection  and  storage  need  to  be  identified 

•  Data  on  local  social  structures 

•  Environmental  data  that  influences  HSCB 

•  Include  medical  data  (e.g.,  HIV  positive) 

•  translating  civilian  attitudes  into  levels  of  cooperation  with  friendly  forces  and  result  in  corresponding 
levels  of  HUMINT  provided  by  the  civilian  population 

•  accounting  for  discrimination  between  civilian  and  adversary  actors  based  on  presented  physical  and 
behavioral  signature  (e.g.,  insurgent  in  civilian  clothing) 

•  accounting  for  how  affiliations  and  support  for  other  actors  change  based  on  the  application  of  influences 
(e.g.,  friendly  operations,  government  activities,  adversary  operations) 

•  accounting  for  changes  in  target  audience  attitudes  caused  by  the  application  of  PSYOPS 

•  reflecting  adversary  HUMINT  networks  (e.g.,  attributes  of  the  HUMINT  network,  how  the  network  is 
formed,  how  the  network  adjusts  if  a  node  or  element  is  removed,  what  adversary  activities  tend  to 
facilitate  or  discourage  the  population’s  provision  of  HUMINT) 

•  accounting  for  unique  adversary  PS  Y OPS  techniques  and  the  effect  of  those  techniques  on  the  target 
audience 
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•  representing  the  effects  of  CMO  on  the  attitudes  of  the  civilian  population  (or  other  target  audience) 

•  the  effects  of  essential  services  (or  lack  thereof)  on  civilian  population’s  attitudes/behaviors 

•  accounting  for  the  effect  of  governmental  corruption  on  governmental  institutions  and  on  the  civilian 
population’s  attitudes/behaviors 

•  reflecting  the  attitudes/behavior  of  actors  (e.g.  civilian  population)  based  on  the  state  of  physical 
infrastructure  used  by  the  actors 

•  accounting  for  non-homogeneous  groups  of  actors.  This  is  particularly  problematic  when  group 
members  have  overlapping  affiliations  (e.g.  a  single  actor  or  group  belongs  to  multiple  groups  -  religious 
groups,  ethnic  groups,  political  groups,  tribal  groups,  etc.) 

•  accounting  for  levels  of  civilian  support  for  and  the  provision  of  physical  or  monetary  resources  to 
adversaries 

•  accounting  for  the  performance/effectiveness  of  actor  organizations  based  on  the  level  of  cooperation 
between  those  organizations  (e.g.  USAID  and  local  government) 

•  accounting  for  the  behavior  of  actors  (e.g.  civilian  population,  religious  leaders)  based  on  their  level  of 
support  for  the  existing  government 

•  accounting  for  the  legitimacy  of  the  existing  government  as  viewed  from  outside  the  nation  by  external 
groups  and  the  effect  of  international  legitimacy  on  government  effectiveness 

•  accounting  for  the  of  the  state  of  the  existing  legal  system  and  its  impact  on  the  attitudes/behaviors  of  the 
civilian  population 

•  civilian  attitudes/behaviors  based  on  existing  economic  conditions  and  how  the  attitudes/behaviors 
change  as  the  economic  conditions  change 

•  the  effects  of  media  activity  on  the  attitudes/behaviors  of  actors 

•  the  effects  of  friendly  operations  on  media  themes  and  activity 

4.3  MODELING  NEEDS 

The  modeling  portion  of  the  HSCB  needs  is  organized  into  Tools,  Representations  in  Tools,  Prediction,  Design  of 
Experiments,  and  VV&A.  VV&A  actually  spans  both  modeling  and  social  sciences;  however,  it  is  placed  here 
for  convenience. 

4.3.1  Tools 

Observation:  There  is  no  such  thing  as  a  “silver  bullet”  model  that  will  satisfy  all  of  our  HSCB  modeling  needs. 
Existing  HSCB  tools  are  too  limited  in  their  scope  (e.g.,  counterinsurgency  tools  can  not  address  multiple 
insurgencies,  simultaneously).  We  must  understand  the  limits  to  prediction  when  addressing  HSCB  issues 

•  We  require  an  expanded  set  of  HSCB  tools 

•  The  proper  architecture/framework  is  an  open  question;  however,  there  are  many  suggestions 

o  The  suite  of  tools  should  include  a  balanced  mix  of  techniques  that  take  advantage  of  the 

strengths  of  the  tools  while  ameliorating  their  weaknesses  (e.g.,  system  dynamic  models;  agent 
based  models;  wargames) 

o  Consideration  should  be  given  to  creating  a  “collaborative  environment”  in  which  selected 
models  can  be  linked/federated  and  evaluated  (particularly  with  respect  to  “precision”)Service 
oriented  architecture 

o  Do  we  want  a  GIG  (global  information  grid)  service  of  DIME/PMESII.  We  want  something  that 
plugs  in  a  question  and  spits  out  an  answer  relative  to  the  population, 
o  Hierarchical  modeling  to  navigate  across  theories,  models  and  levels  of  analysis 
o  Hybrid  models  with  fine  grain  locations  for  addressing  particular  issues 

o  holistic  architecture  (level  TBD)  versus  vignette/scenario  driven.  Which  is  most  appropriate  for 
deriving  needs  for  these  models? 
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o  HSCB  Model  Architecture  requires  flexibility  to  integrate  /  aggregate  domains;  e.g.,  integrate 
social-religion-political  domains  when  dealing  with  the  Muslim  world 
o  Define  /  design  a  family  (framework)  of  HSCB  models  that  can  be  integrated  and  federated  to 
model  specific  scenarios  and  casual  effects 

o  Adopt  a  model  framework  that  can  accommodate  meta-model  and  meta-data  aggregation  and 
disaggregation 

o  A  systems  architecture  so  different  models  can  inform  each  other 
o  Integration  of  data,  models,  and  systems  (how  do  the  4  clubs  interact?) 

■  Design  to  allow  output  of  one  model  (e.g.,  agent-based)  to  feed  the  system  dynamics  of 
another  system  component 

■  Need  mapping  of  data  across  models 

•  Include  hand-shake  across  modeling  modalities 

■  Supporting  data  needs  some  agreed  on  and  clearly  defined  objectives 

■  Need  cross-disciplinary  understanding  across  modelers  and  social  scientists 

■  Need  an  integrated  DIME  model 

o  Need  bits  and  pieces  of  models  that  can  used  by  all  and  then  build  from  there 

■  Framework  for  tailoring  the  architecture 
o  Tailorable  and  adaptable  HSCB  models 

■  Focus  on  identification  of  common  factors  as  a  baseline  for  HSCB  models  (e.g., 
ODU/VMASC  Insurgency  Studies  of  Columbia  and  Nigeria  found  60  of  125  insurgency 
factors  to  be  common) 

■  Easily  change/add  non-common  factors 
o  Models  that  are 

■  Tailorable 

■  Updated  within  a  resource  repository 

■  Complex  and  adaptive  while  running 

o  Architecture  that  supports  PMESII  M&S  interoperability  and  reuse 
o  “Service  bus”  for  data  repositories 

■  Provides  management  and  error  checking 

o  Dynamic  Data  Driven  Application  Systems  with  real-time  forecasting 
o  Need  a  resource  repository  for  HSCB  models  and  data 
o  Need  quantitative  and  qualitative  models 
o  Small  and  focused  models  (not  mega  models) 
o  Support  multi-modal  vision  of  situations 
o  Representation  at  multiple  scales  for  Blue,  Red,  Green 

■  Temporal 

■  Spatial 

■  Demographic 

o  To  make  models  efficient  &  cost  effective.  Minimize  model  development  costs,  runtime  and 
overhead  (admin,  user,  and  developer). 

o  Collaborative  planning  capability  that  spans  neighborhood  to  national  (possibly  multi-national) 
level 

o  Analytical  capability  to  evaluate  balance/  tradeoff  options  and  conduct  sensitivity  analysis 
•  Data  connections 

o  How  to  handle  proprietary  information? 

o  How  to  handle  “personal”  information  (privacy  and  use  of  information)? 

o  Standardized,  interoperable  data.  Needs  to  be  available  on  point  to  the  end  user  and  automatically 
enforced.  Knowledge  of  what  model  does  not  do. 
o  Data  and  models  should  co-evolve 

o  Ability  to  obtain,  organize,  and  access  data  (e.g.,  metadata) 
o  Policy  issues  for  open  data  access 
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o  Data  standards,  data  storage,  data  mining,  data  extraction/discovery  tools 

•  Theory  connections 

o  Model  that  accommodates  all  theories,  or  knows  why  not 
o  To  replicate  real  world  activities  and  instantiate  HSBC  theory /protocols/methods 

•  Metrics  connections 

o  Identify  output  metrics  relevant  to  make  decisions 

o  models  should  measure  outcomes  and  not  inputs.  Not  by  what  you  do,  but  by  what  happens  when 
you  do  it. 

o  Criteria  for  thresholds  to  determine  when  transitions  occur  between  S,  S,  T,  and  R 
o  Criteria  for  distinguishing  friendly  locals  from  hostile 
o  Bounds  for  complex,  adaptive  models 

4.3.2  Representations  in  Tools 

Observation:  The  entire  DIME/PMESII  spectrum  needs  to  be  represented.  However,  each  individual  tool  need 
not  cover  the  entire  spectrum. 

•  model  of  ourselves 

•  Situational  awareness  of  all  parties 

•  feedback  between  micro  and  macro 

•  Scalable  organizational  performance  model  of  social  institutions  [city/town].  (MPICE). 

•  ‘external’  influences  (e.g.,  ever-changing  views/norms,  local/societal  demographics,  and  processes  of 
individuals  and  societies)  that  in  turn  influence  both  physical  and  cognitive  environments. 

•  Representation  of  a  wide  spectrum  of  cultures  and  institutions  (formal  and  informal) 

•  Representation  of  behaviors  during  conflict,  disaster,  etc 

•  Flexibility  in  representing  different  types  of  operations  supporting  U.S.  interests 

•  Representation  of  inter-organizational  dynamics 

•  Representation  of  situational  awareness 

•  Representation  of  dynamic,  competing  views/needs/priorities  by  all  groups 

•  Need  to  design  models  to  be  able  to  handle  more  than  one  insurgency  at  a  time 

o  Non-state  actors;  religious  sects;  criminal  -  May  attack  each  with  a  different  line  (military, 
diplomatic,  economic) 

•  Don’t  use  internal  political  boundaries,  but  use  cells  with  flows  among  cells 

4.3.3  “Prediction” 

Observation:  Not  all  models  need  to  “predict”  things.  “Prediction”  may  mean  likelihoods  of  occurrence  of 
multiple  possible  events.  “Forecasting,”  as  in  forecasting  the  weather,  conveys  the  concept  better  than  does  the 
term  “prediction.” 

•  Generate  insightful  multi- factor  commentaty  and  evaluation 

•  Provide  content-rich  recommendations  for  plans  and  policies 

•  Provide  probabilistic  statements,  screened  by  subject  matter  experts 

•  “Prediction”  should  be  replaced  with  “forecasting,”  with  emphasis  on  possible,  multiple  futures 

•  Hard  prediction  of  events 

•  Soft  prediction  of  likelihoods 

•  Exploration  of  possibilities 

•  Situational  awareness  and  understanding 
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4.3.4  Design  of  Experiments 

Observation:  Many  users  of  HSCB  M&S  do  not  understand  how  to  derive  statistically  meaningful  insights  from 
their  tools. 

•  We  should  draw  on  the  insights  developed  at  NPS  for  M&S  characterized  by  large  numbers  of  variables 
to  characterize  the  interesting  parts  of  response  surfaces  using  innovative  experimental  designs  (e.g., 
focused  fractional  factorial  designs) 

4.3.5  Verification,  Validation  and  Accreditation  (VV&A) 

Observation:  Most  creators  of  HSCB  M&S  and  selectors  of  data  do  not  do  an  adequate  job  of  V&V.  We  need  to 
broaden  the  concept  of  V&V  to  theories  and  selection  of  Subject  Matter  Experts  (SMEs).  Precise  definitions, 
levels  of  accomplishment,  descriptions  of  pitfalls  are  not  more  important  than  getting  started  with  performing 
VV&A.  DIME/PMESII  data  V&V  has  similarities  to  combat  model  data  V&V;  however,  significantly  larger 
complexity  of  the  data  sets  means  the  V&V  is  done  at  the  variable  level  by  [  time  span  &  country  (or  unit  of 
analysis),  conceptual  definition], 

•  We  need  guidelines  that  enable  us  to  perform  V&V  (for  theories,  tools,  data,  SMEs)  credibly,  with 
acceptable  levels  of  resources 

•  Additional  attention  must  be  paid  to  the  accreditation  function 

•  Model  assumptions  should  be  evident 

•  Validation  is  required,  in  part  by  peer  review 

•  Models,  data,  methods,  &  theories  all  need  to  be  characterized  with  tags  (done  with  a  type  of  V&V),  with 
visualization  tool  to  find  desired  entries 

•  Best  practices  guide  for  VV&A  of  models  and  data 

•  How  robust  are  results  against  uncertainty?  Model  incompleteness  -  other  possible  answers. 

•  How  do  modelers  reduce  uncertainty  in  “squishy”  HSCB  models? 

•  How  much  uncertainty  reduction  is  feasible/sufficient? 

•  Subjective  nature  of  models  suggests  data  will  be  required  for  both  V&V  and  to  train  the  model? 

•  HSCB  modeling  would  benefit  from  inclusion  of  Bayesian  analysis  with  SMEs  to  take  into  account  a 
combined  scale/score 

•  Need  a  consistent  approach  to  dealing  with  SME  evaluations 

•  Need  to  identify  /  include  multi-disciplinary  &  cross-disciplinary  experts  when  conducting  model  review 

•  How  do  we  VV&A  tools  in  a  world  of  competitive  analysis 

o  For  example,  joint  output  that  opposes  a  Service  may  be  discredited  by  a  pro-Service  SME 
o  How  to  get  something  useful  to  a  3 -Star  that  will  withstand  scrutiny 

•  Need  for  better  evidence-based  assessment 

•  Need  access  to  social  science  people  who  WANT  to  help 

•  How  often  do  we  update  the  model  and  VV&A  it? 

•  Establish  /  identify  references  &  organizations  to  oversee  (provide)  validated  models  and  data  similar  to 
those  for  warfare  models  (e.g.,  Joint  Munitions  Effectiveness  Model) 

4.4  INDIRECT  NEEDS 

Indirect  needs  include  Education  and  Training  and  the  related  topic  of  Outreach. 

4.4.1  Education  and  Training 

Observation:  It  will  take  high  performance,  compatible,  multidisciplinary  teams  (e.g.,  social  scientists,  operations 
analysts)  to  create  and  employ  HSCB  modeling.  However,  those  diverse  communities  have  a  difficult  time  in 
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communicating.  Communicating  HSCB  model  assumptions  and  results  to  decision  makers  is  even  more  difficult 
than  for  combat  models.  There  is  a  minimal  level  of  continuity  of  interest,  below  which  HSCB  modeling 
initiatives  produce  promising  results,  but  later  are  forgotten.  “Train  as  we  fight”  becomes  much  more  important 
and  difficult  in  HSCB  environment. 

•  Augment  the  curriculum  for  social  scientists  and  operations  analysts  to  give  them  adequate  education  to 
enhance  communication 

•  Bridge  training/education/operations  -  all  tailored  to  be  the  same,  see  book  “Ender’s  Game”. 

•  Create  and  sustain  a  HSCB  Community  of  Interest  (COI)  (perhaps  drawing  on  MORS  Social  Science 
COI)  to  foster  high  performance,  multidisciplinary  teams 

•  Either 

o  construct  converters  of  mathematical  models  to  homilies  or 
o  Educate  policy-makers,  politicians,  general  public  in  mathematical  models 

4.4.2  Outreach 

Observation:  The  HSCB  community  must  include  Interagency  participants.  HSCB  modeling  is  of  interest  to  our 
international  partners. 

•  Expand  the  HSCB  COI  to  include  balanced  interagency  participants  (e.g.,  NSC,  USIP,  DOS,  DOJ) 

•  Participate  in  international  forums  that  address  HSCB  modeling  issues  (e.g.,  NATO  SAS  initiatives  on 
HSCB,  Irregular  warfare) 


4.5  NEXT  STEPS 

This  workshop  was  successful  in  generating  a  coherent  set  of  detailed  results  because  of  careful  planning  and 
execution.  It  will  be  followed  by  two  subsequent  workshops  that  build  on  its  achievements.  In  Workshop  2,  the 
participants  will  be  asked  to  characterize  the  state-of-the-art  in  HSCB  modeling  with  respect  to  the  twelve 
categories  cited  above.  They  will  also  be  asked  to  compare  the  state-of-the-art  to  these  needs  to  identify  major 
gaps.  Although  the  precise  criteria  have  yet  to  be  finalized,  there  is  interest  in  assessing  the  risks  associated  with 
these  gaps  (e.g.,  their  severity  and  frequency). 

In  Workshop  3,  the  participants  will  be  asked  to  assess  the  resources  that  are  needed  to  mitigate  those  gaps. 
Based  on  the  risk  assessments  and  resource  needs,  it  should  be  feasible  to  identify  “low  hanging  fruit”  (e.g., 
significant  gaps  for  which  limited  resources  are  needed)  as  well  as  vital  gaps  (e.g.,  gaps  that  are  catastrophic  or 
critical  and  occur  frequently).  The  results  of  those  deliberations  will  be  used  to  generate  an  initial  version  of  a 
Science  &  Technology  Roadmap  for  HSCB  modeling.  It  is  anticipated  that  this  roadmap  will  be  a  “living 
document”  that  evolves  as  we  gain  a  deeper  understanding  of  the  problem. 

Subsequently,  the  results  of  the  workshops  will  be  used  to  justify  the  allocation  of  resources  for  HSCB  modeling 
to  OSD  and  Congress.  As  necessary,  follow-on  workshops  will  be  conducted  to  characterize  progress  and  to  re¬ 
evaluate  priorities. 
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5.  APPENDIX:  THEORIES  AND  DATA  SOURCES 


Prior  to,  during,  and  after  the  workshop,  the  attendees  supplied  theories  and  data  sources  relevant  to  HSCB 
modeling.  While  not  exhaustive,  the  lists  provide  a  reasonable  starting  point  for  investigations.  The  descriptions 
of  the  theories  and  data  sources  were  often  taken  directly  from  external  sources  and  placed  into  the  database  and 
are  sometimes  quite  lengthy.  To  conserve  space  and  to  avoid  intellectual  property  problems,  only  short  excerpts 
of  the  descriptions  are  printed  here. 
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5.1  THEORIES 


DiseiplineName 

Conceptual  Model 

Theory  Name 

Theory  Description 

Theory  Citation 

Type 

Description  Value 

_None_ 

No  Justification 

Arbitrary  or  accidental  choice 

None 

Nil 

Uncodified 

0 

Not  Modeled 

Not  included  in  the  model 

None 

Nil 

Uncodified 

0 

Anthropology 

AssymetricInfoNoTrus 

Assymetric  Info  /  No  Trust  /  Terroristic 

Violence:  groups  fight  because  of  lack  of  trust, 
terrorize  one  another  to  intimidate  and  gain 

Schneider,  Harold  K.,  1979 

Livestock  and  Equality  in  East 

Africa  :  The  Economic  Basis  for 

Social  Structure.  BLoomington, 
Indiana:  Indiana  University  Press.; 
Bonhage-Freund,  Mary  Theresa, 
and  Jeffrey  A.  Kurland,  1994  Tit- 
for-tat  among  the  Iroquois:  A 

Game  Theoretic  Perspective  on 
Inter-Tribal  Political  Organization. 
Journal  of  Anthropological 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

CircumscriptionTheor 

People  fight  over  resources  that  are  scarce 
relative  to  population  levels 

Cameiro,  Robert  L.,  1970  A 

Theory  of  the  Origin  of  the  State. 
Science  169:733-738.;  2002  Was 
the  Chiefdom  a  Congelation  of 

Ideas?  Social  Evolution  and  History 

PeerRvw 

Peer  Reviewed 
Theory 

3 

CuIturalGroupSelecti 

Cultural  Group  Selection:  only  cultures  with 
people  who  favor  group  survive.  Leads  to  natural 
altruism  toward  in-group,  xenophobia  toward 

Soltis,  Joseph,  Robert  Boyd,  and 

Peter  J.  Richerson,  1995  Can 
Group-functional  Behaviors  Evolve 
by  Cultural  Group  Selection? 

Current  Anthropology  36(3):473- 
494.;  Richerson,  Peter  J.,  and 

Robert  Boyd,  2005  Not  by  Genes 
Alone:  How  Culture  Transformed 
Human  Evolution.  Chicago: 

SWAG+ 

SWAG  with 
some  review 

2.5 

CulturalSuccess 

Men  fight  to  achieve  culturally-defined  success, 
in  order  to  attract  mates 

Chagnon,  Napoleon,  1988  Life 
Histories,  Blood  Revenge,  and 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Warfare  in  a  Tribal  Population. 

Science  239:  985-992. ;  Irons, 

William,  1979  Cultural  and 
Biological  Success.  In  Evolutionary 
Biology  and  Human  Social 
Behavior.  Napoleon  Chagnon  and 
William  Irons,  eds.  Pp.  257-272. 

North  Scituate,  Masachussetts: 

EconomicDefendabilit  Economic  defendability:  people  fight  when  Dyson-Hudson,  Rada,  and  Eric  PeerRvw+  Well  reviewed  3.5 


resources  are  predictable  and  valuable 

Alden  Smith,  1978  Human 
Territoriality:  An  Ecological 
Reassessment.  American 

theory 

FineArt  Development 

superfluity  of  food  resources  makes  possible  the 
development  of  fine  arts 

SWAG+ 

SWAG  with 
some  review 

2.5 

F  raternallnterestGro 

Fraternal  Interest  Groups:  males  in  groups  have 
common  interests  and  fight  to  protect 

Otterbein,  Keith  F.,  1970  The 
Evolution  of  War:  A  Cross- 
Cultural  Study.  New  Haven, 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

ModernizationTheory 

theory  of  social  evolution  closely  related  to 
dependency  theory  and  development  theory: 

Western  countries  are  the  most  developed, 
development  stages  go  from  the  traditional 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Neoevolution 

a  modem  multilineal  cultural  evolution  theory 
that  stresses  empirical  evidence 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Origin  of  Government 

As  farming  populations  gathered  in  larger  and 
denser  communities,  interactions  between 

David  Christian,  p.  245,  Maps  of 

PeerRvw 

Peer  Reviewed 
Theory 

3 

different  groups  increased  and  the  social  pressure 
rose  until,  in  a  striking  parallel  with  star 
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DiseiplineName 

Conceptual  Model 

Theory  Name 

Theory  Description 

Theory  Citation 

Type 

Description  Value 

PopulationDensitF  ood 

Population  Density  -  Food:  when  populations 
increase,  pressure  is  placed  on  food  resources; 

Cohen,  Mark,  1977  The  Food 

Crisis  in  Prehistory.  New  Haven: 

Yale  University  Press. 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

PostindustrialSociet 

industrial  societies  are  coming  to  an  end  and 
services  and  information  are  becoming  more 

SWAG+ 

SWAG  with 
some  review 

2.5 

ProteinHypothesis 

people  in  tribal  societies  fight  over  protein 
sources,  esp.  hunting  territories 

Gross,  Daniel,  1975  Protein 

Capture  and  Cultural  Development 

SWAG+ 

SWAG  with 
some  review 

2.5 

in  the  Amazon  Basin.  American 
Anthropologist  77:526-549.; 

Harris,  Marvin,  1974  Cows,  Pigs, 
Wars,  and  Witches:  The  Riddles  of 
Culture.  New  York:  Random 
House.;  Wilson,  David  J.,  1999 
Indigenous  South  Americans  of  the 
Past  and  Present:  An  Ecological 
Perspective.  Boulder,  Colorado: 


SacredValues 

Conflict  over  meterial  issues  and  rationally 
motivated,  but  sacred  symbols  contested 

Atran,  Scott,  Robert  Axelrod,  and 
Richard  Davis,  2007  Sacred  Barriers 
to  Conflict  Resolution.  Science 

WAG+  WAG  plus 

some  science 

1.5 

Sigmoid  U  tility T  heory 

Individuals  and  groups  of  individuals  with 
common  interest  motivated  by  envy  and  greed 
with  respect  to  others'  social  status 

Friedman,  Milton,  and  Leonard  J. 
Savage,  1 948  The  Utility  Analysis 
of  Choices  Involving  Risk.  Journal 

PeerRvw  Peer  Reviewed 
Theory 

3 

of  Political  Economy  4:279-304.; 
Cancian,  F,  1972  Change  and 
Uncertainty  in  a  Peasant  Economy. 
Stanford:  Stanford  University 
Press.;  Kuznar,  Lawrence  A.,  2001 
Risk  Sensitivity  and  Value  among 
Andean  Pastoralists:  Measures, 
Models  and  Empirical  Tests. 
Current  Anthropology  42(3):432- 
440.;  2002  Evolutionary 
Applications  of  Risk  Sensitivity 
Models  to  Socially  Stratified 
Species:  Comparison  of  Sigmoid, 
Concave  and  Linear  Functions. 
Evolution  and  Human  Behavior 
23(4):265-280.;  2007  Rationality 
Wars  and  the  War  on  Terror: 
Explaining  Terrorism  and  Social 
Unrest.  American  Anthropologist 
109(2):3 18-329.;  Kuznar, 
Lawrence  A.,  and  William  G. 
Frederick,  2003  Environmental 


Socialization 

people  learn  violence 

Whitings  of  HRAF 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Sociobiology 

adapts  neo-Darwinism  to  social  evolution, 
positing  that  humans  are  products  of  both 
biological  evolution  and  sociocultural  evolution 

PeerRvw 

Peer  Reviewed 
Theory 

3 

TradingRaiding 

Intense  interaction,  disputes  over  trade, 
opportunities  to  steal  motivate  violence  between 

Keeley,  Lawrence  H.,  1996  War 
before  Civilization:  The  Myth  of 
the  Peaceful  Savage.  Oxford: 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

TransCulturalDiffusi 

concept  that  ideas,  styles,  religions, 
technologies,  languages,  etc.  spread  between 
individuals  and  thus  through  and  between  cultures 

Accepted 

Generally 

Accepted 

4 

U  nilinealE  volution 

concept  that  all  societies  pass  through  a  single 
evolutionary  process  from  the  most  primitive  to 

WAG 

Wild  Assed 

1 

WomanCapture 

Men  fight  to  capture  women  for  wives;  men 
fight  over  adultery 

Chagnon,  N.,  1977  Yanamamo. 

New  York:  Holt,  Rinehart  and 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Business  Management 


Bureaucracy 

Theory  of  bureacracies:  design  them  so  they 
work  despite  the  individual 

Fayol,  Henri 

SWAG+ 

SWAG  with 
some  review 

2.5 

Parkinson's  Law 

Work  expands  to  fill  the  time  available 

Parkinson,  C.  Northcote 

SWAG+ 

SWAG  with 
some  review 

2.5 

Peter  Principle 

People  rise  to  their  level  of  incompetence 

Peter,  Laurence  J. 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

Satisficing 

decision  making  where  the  first  choice  that  is 
within  the  acceptable  range  is  picked 

PeerRvw 

Peer  Reviewed 
Theory 

3 
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DiseiplineName  Conceptual  Model 


Theory  Name 

Theory  Description 

Theory  Citation 

Type 

Description  Value 

Theory  X 

Management  style  theory:  people  dislike  work 
and  must  be  coerced 

McGregor,  Douglas 

SWAG 

Scientific  Wild 

Assed  Guess 

2 

Theory  Y 

Management  style  theory:  people  seek 

McGregor,  Douglas 

SWAG 

Scientific  Wild 

Assed  Guess 

2 

Computer  Science 

CoherenceThThought 

Coherence  theory  of  thought  and  action 

Rescher,  Nicholas.  1982.  The 
coherence  theory  of  truth. 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Washington,  D.C.:  University  Press 
of  America.;  Rickheit,  Gert  and 
Christopher  Habel.  1995.  Focus  and 
coherence  in  discourse  processing. 
New  York:  W.  de  Gruyer;  Tannen, 
Deborah.  1984.  Coherence  in 
spoken  and  written  discourse. 
Norwood,  N.J.:  ABLEX  Pub.  Corp; 
Thagard,  Paul.  2000.  Coherence  in 
Thought  and  Action.  Cambridge, 


ContinuousSimulation 

Continuous  (actually  time-stepped)  simulation, 
with  time  units  determining  precision 

Various 

PeerRvw+  Well  reviewed 
theory 

3.5 

DiscreteEventSimulat 

Discrete  event  simulation  of  time 

Various 

PeerRvw+  Well  reviewed 
theory 

3.5 

Greedy  Algorithm 

an  heuristic  algorithm  that  at  every  step  selects 
the  best  choice  available  without  regard  to  future 

PeerRvw  Peer  Reviewed 
Theory 

3 

Identical  Variables 

Connection  is  on  a  variable  by  variable  basis, 
with  exact  semantic  correspondence 

None 

Accepted  Generally 

Accepted 

4 

MixedContinDiscrete 

Mixed  continuous  and  discrete  event  simulation 

Various 

Accepted  Generally 

Accepted 

4 

Programmer'sDecisio 

The  programmer  had  to  make  an 

None 

WAG  Wild  Assed 

1 

RandomN  umberGene 

Random  number  generators  do  not  generate  truly 
random  numbers.  The  quality  of  the  generator 
determines  the  nearness  to  true  randomness  and 
the  policies  for  using  the  generators  when  many 

PeerRvw+  Well  reviewed 
theory 

3.5 

Simulated  Annealing 

computational  method  that  is  inspired  by  the 
annealing  process  (cf.  annealing  glass).  Uses  the 

Accepted  Generally 

Accepted 

4 

Static  model 

No  time  representation 

Various 

PeerRvw  Peer  Reviewed 
Theory 

3 

Tabu  Search 

a  metaheuristic  that  guides  a  local  heuristic 
search  procedure  to  explore  the  solution  space 

Accepted  Generally 

Accepted 

4 

Data  Issues 

Likert  Scales 

Economics,  Macro 

Subjective  evaluations  using  Lickert  Sealses 

Various 

PeerRvw  Peer  Reviewed 
Theory 

3 

Economics,  Micro 

Preference  Theory 

studies  the  fundamental  aspects  of  individual 
choice  behavior,  such  as  how  to  identify  and 

PeerRvw+  Well  reviewed 
theory 

3.5 

quantify  an  individual's  preferences  over  a  set  of 
alternatives  and  how  to  construct  appropriate 
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DiseiplineName 
Theory  Name 

Prospect  Theory 

Supply  and  Demand 
Utility  Theory 

Education 

BehaviorConditioning 

Constructivism 

OperantConditioning 

SocialCognitiveLearn 

Engineering,  Civil 

Engineering,  Other 

Control  Theory 

Geography,  Human 

Geography,  Physical 

Arc-Node  network 

DTED-0 

DTED-1 

DTED-2 

Hex  or  Square  grid 
Lat-Long  coordinates 
No  spatial  coords 
XY  coordinates 

History 

DemographicStructur 

Law 


Conceptual  Model 


Theory  Description 

Theory  Citation 

Type 

Description  Value 

A  prospect  is  a  contract  {xl,pl;...;xn,pn}  that 
yields  outcome  xi  with  probability  yi.  Prospect 
theory  is  a  descriptive  theory  of  choice 
explaining  common  attitudes  toward  risk  that 

Daniel  Kahneman  and  Amos 

Tversky.  Prospect  theory:  an 
analysis  of  decision  under  risk. 
Econometrica,  47(2):263-292, 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Supply  versus  Demand  Curve 

Various 

Accepted 

Generally 

Accepted 

4 

study  of  preferences  structures  and  the  ways  to 
represent  the  preferences  quantitatively 

Accepted 

Generally 

Accepted 

4 

Classical  conditioning  (sometimes  referred  to  as 
Pavlovian  conditioning)  is  a  technique  used  in 
behavioral  training  in  which  a  naturally  occurring 
stimulus  is  paired  with  a  response.  Next,  a 

Pavlov,  Skinner 

PeerRvw 

Peer  Reviewed 
Theory 

3 

learning  is  an  active  process  in  which  learners 
construct  new  ideas  or  concepts  based  upon  their 
current/past  knowledge.  The  learner  selects  and 
transforms  information,  constructs  hypotheses, 

Bruner,  1960,  1966,  Piaget,  Vgotsky 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Operant  conditioning  (sometimes  referred  to  as 
instrumental  conditioning)  is  a  method  of 
learning  that  occurs  through  reinforcement  and 
punishments  for  behavior.  Through  operant 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Social  Cognitive  (Learning)  Theory  (Bandura)— 
Theory  posits  that  people  learn  from  one 
another,  via  observation,  imitation,  and 
modelling  and  do  not  necessarily  need  direct 

Bandura 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Differential  &  difference  equations,  stochastic 
processes,  etc.,  to  steer  dynamical  systems 

Accepted 

Generally 

Accepted 

4 

Planar  representation  of  significant  points 

Various 

SWAG+ 

SWAG  with 
some  review 

2.5 

Digital  Terrain  Elevation  Data  0  is  about  1 

US  Govt 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Digital  Terrain  Elevation  Data  1  is  about  100 

US  Govt 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Digital  Terrain  Elevation  Data  2  is  about  30 

US  Govt 

Accepted 

Generally 

Accepted 

4 

Planar  representation  depending  on  size  of  grid 

Various 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Planar  dimensions  taking  acount  of  Earth's  shape 

Various 

Accepted 

Generally 

Accepted 

4 

No  spatial  representation 

Various 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

Planar  dimensions  with  good  local  representation 

Various 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Demographic-structural  theory:  Population 
growth  beyond  the  means  of  subsistence  leads  to 
declining  levels  of  consumption  and  popular 
discontent,  but  this  is  not  enough  to  destabilize 

Turchin,  Peter,  "Arise 
'cliodynamics',"  Nature,  Vol  454, 

No  3,  3  July  2008;  Turchin,  P.  &  S. 
Nefedov,  Secular  Cycles,  Princeton 

SWAG 

Scientific  Wild 
Assed  Guess 

2 

DiseiplineName 
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Conceptual  Model 

Theory  Name  Theory  Description 


Theory  Citation 


Type  Description  Value 


Linguistics 

Discourse  Theory  A  branch  of  ethnomethodology  focusing  on  Blommaert,  Jan.  2005.  Discourse:  Accepted  Generally  4 

interactive  communication.  A  Critical  Introduction.  New  York:  Accepted 

Cambridge  University  Press; 

Tannen,  Deborah  and  James  E. 

Alatis.  2003.  Linguistics,  language 
and  the  real  world  :  discourse  and 

SemanticDifferential  Osgood's  semantic  differential  was  designed  to  Osgood,  C.E.,  Suci,  G.,  &  PeerRvw  Peer  Reviewed  3 

measure  the  connotative  meaning  of  concepts.  Tannenbaum,  P.  (1957)  The  Theory 

The  respondent  is  asked  to  choose  where  his  or  measurement  of  meaning.  Urbana, 

her  position  lies,  on  a  scale  between  two  bipolar  IL:  University  of  Illinois  Press 

Snider,  J.  G.,  and  Osgood,  C.  E. 

(1969)  Semantic  Differential 
Technique:  A  Sourcebook.  Chicago: 

Aldine.  Osgood,  C.  E.,  May,  W. 

H.,  and  Miron,  M.  S.  (1975)  Cross- 
Cultural  Universals  of  Affective 
Meaning.  Urbana,  IL:  University  of 
Illinois  Press  Krus,  D.J.,  & 

Ishigaki,  Y.  (1992)  Kamikaze 
pilots:  The  Japanese  and  the 


Medicine 


DALY  Model 

Disability  Adjusted  Life  Years  (DALY).  DALYs 
measure  the  gap  between  the  average  person’s 
life  across  many  different  levels  of  health  and  a 
theoretical  perfectly  healthy  life.  The  gap  is 

Mathers  CD,  Vos  T,  Lopez  AD, 
Salomon  J,  Ezzati  M  (ed.)  2001. 
National  Burden  of  Disease 

Studies  A  Practical  Guide.  Edition 

2.0  Global  program  on  Evidence  for 
Health  Policy.  Geneva:  World 

SWAG+  SWAG  with 
some  review 

2.5 

SIR  Model 

SIR  (Susceptible  -  Infectious  -  Recovered) 
model. SIR  model  illustrates  the  transition  of  a 
healthy  person  to  a  sick  person  to  a  dead  person 
(or  recovered  person)  by  a  generic  disease 

W.  O.  Kermack  and  A.  G. 

PeerRvw  Peer  Reviewed 
Theory 

3 

SIS  Model 

Susceptible-Infectious-Susceptible  (SIS)  models 
adapted  from  the  SIR  model 

SWAG+  SWAG  with 
some  review 

2.5 

Meteorology 

Meteorology 

Effects  of  atmospheric  conditions  on  general 

Various 

SWAG+  SWAG  with 
some  review 

2.5 

Military  Science 

BonderFarrell 

Bonder-Farrel  attrition  at  the  item  level 

Bonder  &  Farrell 

PeerRvw+  Well  reviewed 
theory 

3.5 

FM  3-24 

Counterinsurgency  manual 

US  Army  12/06 

PeerRvw+  Well  reviewed 
theory 

3.5 

HartleyLogLinearLaw 

Attrition  at  the  aggregated  level 

Hartley 

PeerRvw  Peer  Reviewed 
Theory 

3 

LanchesterLinearLaw 

"Area  Fire"  attrition  at  the  aggregated  level 

Lanchester 

SWAG+  SWAG  with 
some  review 

2.5 

LanchesterSquareLaw 

"Aimed  Fire"  attrition  at  the  aggregated  level 

Lanchester 

SWAG+  SWAG  with 
some  review 

2.5 

Small  Wars  Manual 

Counterinsurgency 

USMC  1940 

PeerRvw+  Well  reviewed 
theory 

3.5 
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DiseiplineName 

Conceptual  Model 

Theory  Name 

Theory  Description 

Theory  Citation 

Type 

Description  Value 

Operations  Research 

A* 

A-star  search  algorithm  finds  the  least-cost  path 
from  a  given  initial  node  to  one  goal  node  (out 
of  one  or  more  possible  goals).  It  uses  a 
distance-plus-cost  heuristic  function  (usually 

1968  by  Peter  Hart,  Nils  Nilsson, 
and  Bertram  Raphael 

Accepted 

Generally 

Accepted 

4 

AHP 

Analytic  Hierarchy  Process  derives  ratio  scales 
from  both  discrete  and  continuous  paired 
comparisons  in  multilevel  hierarchic  structures. 

Saaty 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Bayes'  Theorem 

Bayes'  theorem  relates  the  conditional  and 
marginal  probabilities  of  events  A  and  B,  where 

B  has  a  non-vanishing  probability: 

various 

Accepted 

Generally 

Accepted 

4 

BayesianDecisionThe 

Uses  Bayes  Rule  to  update  probability  based  on 
new  evidence 

Accepted 

Generally 

Accepted 

4 

BoundedRationality 

A  theory  of  choice  making  that  says  the  reason 
people  don't  make  fully  rational  choices  is  they 
lack  the  knowledge  and  computational  skill  to  do 

Simon,  1955,  1979;  Gigerenzer, 

Gerd  and  Reinhard  Selten.  2002. 
Bounded  Rationality:  The 

Adaptive  Toolbox.  Cambridge,  MA: 
MIT  Press;  Rubinstein,  Ariel. 

1997.  Modeling  Bounded 

Rationality.  Cambridge,  MA:  MIT 
Press;  Simon,  Herbert  Alexander. 

1982.  Models  of  bounded 
rationality.  Cambridge,  MA:  MIT 

Accepted 

Generally 

Accepted 

4 

Cluster  Analysis 

A  generic  term  for  various  procedures  used  to 
objectively  group  entities  based  on  similarities 

Accepted 

Generally 

Accepted 

4 

CostEffectivenessAna 

Cost  Effectiveness  Analysis  compares  costs, 
benefits  and  risks  of  alternative  solutions. 

Accepted 

Generally 

Accepted 

4 

DEA 

Data  Envelopment  Analysis  (DEA)  is  finding  the 
efficient  frontier  of  economic  alternatives 

Accepted 

Generally 

Accepted 

4 

Decision  Tree 

pictorial  description,  nodes  &  arcs,  of 
connections  among  decisions  and  chance  events 
-  must  be  tree  shaped.  Evalutes  to  best  expected 

Accepted 

Generally 

Accepted 

4 

Dempster- 

The  Dempster-Shafer  theory  is  a  mathematical 
theory  of  evidence[l]  based  on  belief  functions 
and  plausible  reasoning,  which  is  used  to  combine 
separate  pieces  of  information  (evidence)  to 

various 

Accepted 

Generally 

Accepted 

4 

Dijkstra's  algorithm 

For  a  given  source  vertex  (node)  in  the  graph, 
the  algorithm  finds  the  path  with  lowest  cost 
(i.e.  the  shortest  path)  between  that  vertex  and 
every  other  vertex.  It  can  also  be  used  for 

Edsger  Dijkstra  in  1959 

Proved 

Scientifically 

Proved  Theory 

5 

DynamicProgrammin 

both  an  approach  to  problem  solving  and  a 
decomposition  technique  where  there  is  a 
sequence  of  interrelated  decisions.  Uses  recursive 

Bellman,  1957 

Accepted 

Generally 

Accepted 

4 

ExponentialSmoothin 

a  technique  for  revising  an  estimate  of  the 
average  of  a  time  series  to  extrapolate  as  a 
forecast.  New  forecast=(l-alpha)*old  forecast  + 

Accepted 

Generally 

Accepted 

4 

Game  Theory 

various  types  of  games,  with  various  rules 

Bennett,  Peter  G.  1995.  "Modeling 
decisions  in  international  relations: 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Game  theory  and  beyond." 

Mershon  International  Studies 
Review  39:19-52;  Neumann,  John 
Von  and  Oskar  Morgenstem.  1944. 

Theory  of  Games  and  Economic 
Behavior.  Princeton,  NJ:  Princeton 
University  Press;  Parsons,  Simon 
and  Michael  Wooldridge.  2002. 

"Game  theory  and  decision  theory 
in  multi-agent  systems." 

Autonomous  Agents  and  Multi- 
Agent  Systems  5:243-254;  Vane, 

Russell  R.  200 1 .  "Hypergame 
theory  for  DTGT  agents."  in  Game 
Theoretic  and  Decision  Theoretic 
Agents:  AAAI  Spring  Symposium, 

Graph  Theory  study  of  interconnetions  of  various  elements  PeerRvw+  Well  reviewed  3.5 

theory 

HiddenMarkovModels  statistical  models  comprising  a  Markov  chian  PeerRvw+  Well  reviewed  3.5 

whose  states  are  associated  with  some  probability  theory 

distributions  -  transition  probabilities 
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Influence  Diagram 

equivalent  to  symmetric  decision  tree 

Accepted  Generally 

Accepted 

4 

Linear  Programming 

Optimal  selection 

Dantzig 

Proved  Scientifically 

Proved  Theory 

5 

Little's  Law 

relates  the  number  of  customers  in  a  queueing 
system  to  the  waiting  time  of  customers  for  a 

Proved  Scientifically 

Proved  Theory 

5 

Markov  Chains 

a  Markov  Process  whose  state  space  is  discrete 
while  its  time  domain  may  be  either  continuous 
of  discrete  (Markov  property  means  that 
probabilities  of  future  events  are  completely 

Proved  Scientifically 

Proved  Theory 

5 

MAUT 

Multi- Attribute  Utility  Theory  (MAUT)  is  useful 
in  quantifying  the  relative  attractiveness  of 
alternatives  with  multiple  attributes,  each  with 

Accepted  Generally 

Accepted 

4 

MCDM 

Multiple  Criteria  Decision  Making  (MCDM) 
includes  deterministic  and  stochastic  outcomes 
and  explicit  (enumerated)  decision  alternatives 
or  implicit  (alternatives  specified  by  constraints) 

PeerRvw+  Well  reviewed 
theory 

3.5 

Neural  Networks 

computational  models  inspired  by  networks  of 

PeerRvw+  Well  reviewed 
theory 

3.5 

Portfolio  Theory 

method  of  selecting  a  portfolio  of  options  using 
economic  concepts 

PeerRvw+  Well  reviewed 
theory 

3.5 

Queueing  Theory 

Derivation  of  wait  times,  service  times,  etc. 
from  structure  of  queues 

various 

Proved  Scientifically 

Proved  Theory 

5 

RPD  and  ImageTheory 

Both  Recognitions-Primed  Decision  Making 
(RPD)  and  Image  theory  are  models  of  choice 
making  involving  pre-existing  knowledge 
structures  against  which  situations  are  evaluated 

Klein,  1993;  Beach,  1990,  1993 

PeerRvw+  Well  reviewed 
theory 

3.5 

Search  Theory 

Theory  of  how  to  search  for  an  object  and 
expected  results  of  the  search 

Search  and  Detection,  Alan  R. 
Washburn,  MAS,  ORSA  (now 

Proved  Scientifically 

Proved  Theory 

5 

Time  Series  Analysis 

most  popular  time  series  forecasting  method  is 

Box,  Jenkins  and  Reinsel  (1994), 

Pandit  &  Wu  (1974) 

Accepted  Generally 

Accepted 

4 

Political  Science 

ContingencyTheory 

conflict  is  contingent  on  unusual  or  irregular 
conditions  that  cause  disruptions  in  conventional 
politics.  The  contingent  approach  leads  one  to 
study  the  political,  economic  and  social 

Harry  Eckstein.  “Theoretical 
Approaches  to  Explaining 

Collective  Political  Violence.”  In 

T.R.  Gurr  (ed.)  Handbook  of 

Political  Conflict,  pp.  135-166. 

SWAG+  SWAG  with 
some  review 

2.5 

InherencyTheory 

The  inherent  perspective  assumes  that  violent 
political  conflict  emerges  out  of  low-level 
contentious  interactions  among  a  set  of  political 
players.  This  approach  leads  researchers  to  focus 

Harry  Eckstein.  “Theoretical 
Approaches  to  Explaining 

Collective  Political  Violence.”  In 

T.R.  Gurr  (ed.)  Handbook  of 

Political  Conflict,  pp.  135-166. 

SWAG+  SWAG  with 
some  review 

2.5 

Psychology 

Additude  Functions 

persuasion  is  highest  when  the  message  address 
the  attitude  function  of  listener  (e.g.,  utilitarian 
function,  social  identity  function) 

DeBono,  1987;  Prentice,  1987; 

Shavitt,  1990,  Shavitt  et  al.,  1992 

PeerRvw  Peer  Reviewed 
Theory 

3 

AffectControlTheory 

Generalizations:  1)  actors  generate  feelings 
appropriate  to  a  social  situation,  2)  actors  who 
can’t  maintain  their  feelings  redefine  their  views 

Heise,  David  R.  1979. 

Understanding  Events:  Affect  and 
the  Construction  of  Social  Action. 

New  York:  Cambridge  University 
Press.;  — .  2006.  Expressive  Order: 
Confirming  Sentiments  in  Social 
Actions.  New  York:  Springer; 
MacKinnon,  Neil  J.  1994.  Symbolic 
Interactionism  as  Affect  Control. 
Albany,  NY:  SUNY  Press;  Smith- 
Lovin,  Lynn.  1988.  "Affect 
control  theory:  An  assessment."  in 
Analyzing  Social  Interaction: 

Advances  in  Affect  Control 

Theory,  edited  by  L.  Smith-Lovin 

PeerRvw+  Well  reviewed 
theory 

3.5 

AttributionTheoryl 

Attribution  theory  is  concerned  with  how 
individuals  interpret  events  and  how  this  relates 
to  their  thinking  and  behavior.  Attribution 
theory  assumes  that  people  try  to  determine  why 

Heider,  1958 

SWAG+  SWAG  with 
some  review 

2.5 

DisciplineName  Conceptual  Model 
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Theory  Name 

Theory  Description 

Theory  Citation 

Type 

Description  Value 

AttributionTheory2 

Kelley  expanded  on  Heider’s  model  and 
attempted  to  explain  the  cognitive  process  by 
which  people  generate  internal  or  external 
explanations  Kelley  proposed  that  individuals 

Kelley,  1967 

SWAG+ 

SWAG  with 
some  review 

2.5 

AttributionTheory3 

Weiner  added  an  additional  dimension  to  causal 
interpretation  when  he  proposed  that  the 
stability  of  the  cause  is  also  included  in 
individual’s  explanations  of  outcomes.  The 

Weiner,  1971 

SWAG+ 

SWAG  with 
some  review 

2.5 

BehavioralTheory 

Behavioral  Theory  (Skinner).  Behaviorism  is  a 
theory  of  learning  based  upon  the  idea  that  all 
behaviors  are  acquired  through  conditioning. 
Conditioning  occurs  through  interaction  with  the 

Skinner 

PeerRvw 

Peer  Reviewed 
Theory 

3 

BigFiveFactorModel 

Started  as  Neuroticism-Extroversion-Openness 
Inventory  (NEO-I),  later  revised  as  NEO 

Personality  Inventory  (NEO  PI-R).  Claims 
personality  encompasses  five  major  traits  (with 

Robert  McCrae  &  Paul  Costa, 

1970s;  Goldberg,  1994;  critique 

SWAG+ 

SWAG  with 
some  review 

2.5 

CognitiveDissonanceT 

Cognitive  Dissonance  Theory  (Festinger)  Social 
psychologist  Leon  Festinger  first  proposed  the 
theory  in  1957  after  the  publication  of  his  book 

When  Prophecy  Fails,  observing  the 

http://en.wikipedia.org/wiki/Cogniti 
ve  dissonance,  May  2008.  Leon 
Festinger.  A  Theory  of  Cognitive 
Dissonance.  Stanford  University 

SWAG+ 

SWAG  with 
some  review 

2.5 

ElaborationLikelihoo 

Elaboration  Likelihood  Model  (ELM)  posits  that 
attitude  change  can  occur  through  either  of  tw 
different  processing  routes:  central  or  peripheral. 
Central  route  processing  produces  attitude 

Petty  &  Cacioppo,  1986;  Petty  & 
Wegener,  1999 

PeerRvw 

Peer  Reviewed 
Theory 

3 

Ethnomethodology 

Empirically-oriented  study  of  microinteraction 
and  its  conceptual  framework. 

Garfinkel,  Harold.  2006.  Seeing 
Sociologically:  The  Routine 

Grounds  of  Social  Action.  Boulder, 

CO:  Paradigm  Publishers;  Heritage, 
John.  1993.  Garfinkel  and 
Ethnomethodology.  Oxford,  UK: 
Blackwell  Publishing;  Hilbert, 

Richard  A.  1992.  The  Classical 

Roots  of  Ethnomethodology: 

Durkheim,  Weber  and  Garfinkel. 
Chapel  Hill,  NC:  University  of 

PeerRvw 

Peer  Reviewed 
Theory 

3 

HeuristicSystematicM 

Heuristic- Systematic  Model  (HSM)  of  social 
information  processing  divides  message 
processing  (for  persuasion)  into  systematic  and 
heuristic  modes  and  describes  what  triggers  each 

Chaiken,  1980,  1987 

PeerRvw 

Peer  Reviewed 
Theory 

3 

InoculationTheory 

Inoculation  theory  concerns  conferring 
resistance  to  persuation. 

McGuire;  Jamieson,  1992 

SWAG+ 

SWAG  with 
some  review 

2.5 

InvolvementELM 

Extension  of  Elaboration  Likelihood  Model 
(ELM)  adds  the  effect  of  induced  involvement  in 

Slater 

PeerRvw 

Peer  Reviewed 
Theory 

3 

LanguageExpectancyTh 

Language  Expectancy  Theory  (for  modeling 
persuasion)  is  an  axiomatic  theory  that  expounds 
on  the  effects  of  linguistic  variations  on 
message  persuasiveness.  It  is  a  message-centered 

M.  Burgoon,  1995 

PeerRvw 

Peer  Reviewed 
Theory 

3 

MMPI 

Minnesota  Multiphasic  Personality  Inventory 
(MMPI).  A  new  and  psychometrically  improved 
version  of  the  MMPI-2  has  recently  been 
developed  employing  rigorous  statistical  methods 

Starke  R.  Hathaway,  PhD,  and  J.  C. 
McKinley,  MD,  originators,  many 
contributors  since 

PeerRvw 

Peer  Reviewed 
Theory 

3 
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Theory  Citation 
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Myers-Briggs 

Myers-Briggs  Type  Indicator  (MBTI)  assessment 
is  a  psychometric  questionnaire  designed  to 
identify  certain  psychological  differences 
according  to  the  typological  theories  of  Carl 

The  Manual 

SWAG+ 

SWAG  with 
some  review 

2.5 

Prototype  Concepts 

The  empirical  discovery  that  human  concepts 
are  organized  radially,  from  the  empirically 

Neisser,  Ulric.  1987.  Concepts  and 
Conceptual  Development: 

Ecological  and  Intellectual  Factors 
in  Categorization.  New  York: 
Cambridge  University  Press;  Rosch, 
Eleanor.  1978.  "Principles  of 
categorization."  Pp.  27-48  in 

Cognition  and  Categorization, 
edited  by  E.  Rosch  and  B.  B.  Lloyd. 
Hillsdale,  NJ:  Lawrence  Erlbaum.; 

— .  1983.  "Prototype  classification 
and  logical  classification."  Pp.  73- 
86  in  New  Trends  in  Conceptual 
Representation:  Challenges  to 

Piaget’s  Theory?,  edited  by  E.  K. 
Scholnick.  Hillsdale,  NJ:  Lawrence 
Erlbaum;  Wittgenstein,  Ludwig. 

1958.  Philosophical  Investigations 
:  the  English  text  of  the  third 

Accepted 

Generally 

Accepted 

4 

ReactanceTheory 

Reactance  Theory.  It  operates  in  three  simple 
steps  that  are  sequentially  connected.  People 
perceive  an  unfair  restriction  on  their  actions.  A 
state  of  reactance  is  activated.  Reactance  is  an 

Brehm,  1966 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-affirmationTheo 

Self-affirmation  theory.  This  theory  explains 
how  people  will  reduce  the  impact  of  a  threat  to 
their  self-concept  by  focusing  on  and  affirming 
their  competence  in  some  other  area. 

Steele,  1988 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-awarenessTheory 

Duval  &  Wickland,  1972 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-completionTheor 

Self-completion  theory.  Symbolic  self 
completion  theory  argues  that  many  of  the 
activities  that  individuals  enact — such  as 
possessions  they  purchase — are  intended  to 

Wicklund  &  Gollwitzer,  1982 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-discrepancyTheo 

Self-discrepancy  theory.  We  are  strongly 
motivated  to  maintain  a  sense  of  consistency 
among  our  various  beliefs  and  self-perceptions. 

This  causes  problems  as  there  are  invariably 

Higgins  et  al.,  1987 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-evaluatnMainten 

Self-evaluation  maintenance  theory.  The  theory 
suggests  that  one’s  self-concept  can  be 
threatened  by  how  other  people  behave.  The 
level  of  threat  depends  both  on  how  close  the 

Tesser,  1988 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-monitoringTheor 

Self-monitoring  theory.  The  theory  refers  to  the 
process  through  which  people  regulate  their  own 
behavior  in  order  to  "look  good"  so  that  they 
will  be  perceived  by  others  in  a  favorable 

Snyder,  1974 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-perceptionTheor 

Self-perception  theory.  It  asserts  that  we 
develop  our  attitudes  by  observing  our  own 
behavior  and  concluding  what  attitudes  must 
have  caused  them.  Self-perception  theory  differs 

Bern,  1972 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-presentationThe 

Self-presentation  theory.  Impression 
management  is  the  process  through  which  people 
try  to  control  the  impressions  other  people 
form  of  them.  It  is  a  goal-directed  conscious  or 

Goffman,  1959 

SWAG+ 

SWAG  with 
some  review 

2.5 

Self-verificationThe 

Self-verification  theory.  The  theory  asserts  that 
people  want  to  be  known  and  understood  by 
others  according  to  their  firmly  held  beliefs  and 
feelings  about  themselves.  Because  chronic  self- 

Swann,  1990 

SWAG+ 

SWAG  with 
some  review 

2.5 

SocialCognitiveTheor 

Social  Cognitive  Theory  (formerly  social 
learning  theory).  -  Social  cognitive  theory 
provides  a  framework  for  understanding, 
predicting,  and  changing  human  behavior.  The 

Bandura  1977;  Bandura  1986 

PeerRvw 

Peer  Reviewed 
Theory 

3 

SocialComparison 

The  theory  that  people  evaluate  their  opinions 
and  abilities  in  comparison  to  other  people. 

Many  opinions  and  abilities  cannot  be  evaluated 
without  some  sort  of  reference.  The  theory 

Leon  Festinger.  A  theory  of  social 
comparison  process.  Human 

Relations,  pages  117-140,  1954. 

SWAG+ 

SWAG  with 
some  review 

2.5 
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Theory  Citation 
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SocialExchangeTheory 

Social  Exchange  theory  explains  how  we  feel 
about  a  relationship  with  another  person  as 
depending  on  our  perceptions  of:  a)  the  balance 
between  what  we  put  into  the  relationship  and 

Thibaut  &  Kelly,  1959 

SWAG+ 

SWAG  with 
some  review 

2.5 

SocialF  acilitation 

Social  facilitation.  Tasks  which  we  find  easy,  are 
done  *better/quicker*  in  the  presence  of  other 
people.  The  opposite  is  true  for  difficult  tasks. 

This  is  because  first,  the  presence  of  others 

Tripp,  1918;  Zajonc 

SWAG+ 

SWAG  with 
some  review 

2.5 

TerrorMgmtTheory 

Terror  Management  Theory.  It  is  based  on 

Ernest  Becker's  writings,  in  which  the 
universality  of  death  terror  and  the  need  to 
protect  against  it  play  an  essential  role. 

Greenberg,  Solomon,  & 

SWAG+ 

SWAG  with 
some  review 

2.5 

TheoryReasonedActio 

Theory  of  Reasoned  Action  (TRA)  was 
developed  to  explain  volitional  behaviors.  It 
excludes  spontaneous,  impulsive,  habitual, 
cravings  driven,  scripted  or  mindless  behaviors. 

Fishbein  &  Ajzen,  1975,  1980 

PeerRvw 

Peer  Reviewed 
Theory 

3 

ThOfPlannedBehavior 

Theory  of  Planned  Behavior.  Theory  of 

Reasoned  Action  suggests  that  a  person's 
behavior  is  determined  by  his/her  intention  to 
perform  the  behavior  and  that  this  intention  is, 

Ajzen  &  Fishbein,  1980 

PeerRvw 

Peer  Reviewed 
Theory 

3 

T  riangularThOfLo  ve 

The  triangular  theory  of  love.The  theory 
characterizes  love  within  the  context  of 
interpersonal  relationships  by  three  different 
components:  intimacy,  passion  commitment. 

Sternberg,  1986 

SWAG+ 

SWAG  with 
some  review 

2.5 

TwoFactorTheoEmotio 

Two-factor  theory  of  emotion.  When  trying  to 
understand  what  kind  of  person  we  are,  we  first 
watch  what  we  do  and  feel  and  then  deduce  our 
nature  from  this.  This  means  that  the  first  step 

Schachter  &  Singer,  1962 

SWAG+ 

SWAG  with 
some  review 

2.5 

Science,  Biology 

Evolution 

Evolutionary  Theory  widely  held  notion  that  all 
life  is  related  and  has  descended  from  a  common 
ancestor:  the  birds  and  the  bananas,  the  fishes 
and  the  flowers  —  all  related.  Darwin's  general 

Darwin 

Accepted 

Generally 

Accepted 

4 

Science,  Physical 

Sociology 

Arousal  Theory 

Arousal  Theory,  cost-reward  theory  suggests  that 
people  feel  upset  when  they  see  a  person  in  need 
and  are  motivated  to  do  something  to  reduce  the 
unpleasant  arousal.  People  then  weigh  the  costs 

SWAG+ 

SWAG  with 
some  review 

2.5 

Comp.Org.Theory 

Discipline  focuses  on  theorizing  about, 
describing,  understanding,  and  predicting  the 
behaviors  of  organizations  and  the  process  of 
organizing  using  formal  approaches 

Kathleen  M.  Carley  and  Lee  Gasser. 
Multiagent  systems:  a  modem 
approach  to  distributed  artificial 
intelligence,  chapter  Computational 
Organization  Theory.  MIT  Press, 

SWAG+ 

SWAG  with 
some  review 

2.5 

Conflict  Theory 

Conflict  theory  argues  that  society  is  not  about 
solidarity  or  social  consensus  but  rather  about 
competition.  Society  is  made  up  of  individuals 
competing  for  limited  resources  (e.g.,  money, 

Conflict  theory  was  elaborated  in 
the  United  Kingdom  by  Max 

Gluckman  and  John  Rex,  in  the 

United  States  by  Lewis  A.  Coser  and 
Randall  Collins,  and  in  Germany  by 
Ralf  Dahrendorf,  all  of  whom  were 
influenced  by  Karl  Marx,  Ludwig 
Gumplovicz,  Vilfredo  Pareto,  Georg 
Simmel,  and  other  founding  fathers 
of  European  sociology. 

PeerRvw+ 

Well  reviewed 
theory 

3.5 

Forced  Migration 

people  place  tremendous  value  on  their  physical 
security  and  will  abandon  their  property  and 
relocate  if  they  feel  their  security  is  threatened 

Davenport,  Christina  A.,  Will  H. 

Moore,  and  Steven  C.  Poe.  2003. 
Sometimes  you  just  have  to 

leave:  Threat  and  refugee 
movements,  1964-  1989. 

SWAG+ 

SWAG  with 
some  review 

2.5 

ImpressionManageme 

An  extension  of  role  theory,  impression 
management  is  both  a  theory  and  process.  The 
theory  argues  that  people  are  constantly  engaged 
in  controlling  how  others  perceive  them.  The 

Erving  Goffman,  1959 

PeerRvw 

Peer  Reviewed 
Theory 

3 

InformtnlSciallnflnc 

Informational  social  influence  occurs  when  one 
turns  to  the  members  of  one's  group  to  obtain 
accurate  information.  A  person  is  most  likely  to 
use  informational  social  influence  in  three 

Sherif,  1935;  Cialdini,  1993 

SWAG+ 

SWAG  with 
some  review 

2.5 
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DiseiplineName 
Theory  Name 

Integration  Theory 

Manifest&LatentFunc 


Micro-Generation 


NormativeSciallnflnc 

Public  discourse 


Relative  Deprivation 


Role  Theory 

SocialConstructionis 


Theory  Description 

Recently,  some  sociologists  have  been  taking  a 
different  approach  to  sociological  theory  by 
employing  an  integrationist  approach  - 
combining  micro-  and  macro-level  theories  to 
Says  there  is  a  difference  between  manifest  and 
latent  functions,  where  manifest  functions  are 
the  intended  functions  of  a  phenomenon  in  a 
social  system  and  latent  functions  are  the 
The  theory  that  social  interaction  at  all  scales  is 
ultimately  microinteraction. 


Normative  social  influence.  Normative  social 
influence  occurs  when  one  conforms  to  be  liked 
or  accepted  by  the  members  of  the  group. 

Study  of  the  beliefs  &  ideologies  in  history  and 
public  policy,  their  dynamics  and  the  resources 
that  shape  those  dynamics. 


Relative  deprivation  is  the  experience  of  being 
deprived  of  something  to  which  one  thinks  one 
is  entitled.  Schaefer  defines  it  as  "the 
conscious  experience  of  a  negative  discrepancy 
Role  Theory  posits  that  human  behavior  is 
guided  by  expectations  held  both  by  the 
individual  and  by  other  people.  The  expectations 
correspond  to  different  roles  individuals  perform 
Social  constructionism  is  a  school  of  thought 
introduced  into  sociology  by  Berger  and 
Luckmann.  Social  constructionism  aims  to 
discover  the  ways  that  individuals  and  groups 


Theory  Citation 

Ritzer  &  Goodman  2004 


Merton,  1957 


Collins,  Randall.  1981.  "On  the 
microfoundations  of 
macrosociology."  American  Journal 
of  Sociology  86:984-1014;  Hilbert, 
Richard  A.  1990. 

"Ethnomethodology  and  the  micro¬ 
macro  order."  American 
Sociological  Review  55:794-808; 
Rawls,  Anne  Warfield.  1987.  "The 
interaction  order  sui  generis: 
Goffman's  contribution  to  social 
theory."  Sociological  Theory 
5:136-149;  Stinchcombe,  Arthur  L. 
2001.  When  Formality  Works: 
Authority  and  Abstraction  in  Law 
Asch,  1951 


Apter,  David  E.  and  Tony  Saich. 
1994.  Revolutionary  Discourse  in 
Mao's  Republic.  Cambridge,  MA: 
Harvard  University  Press; 
Converse,  Phillip.  1964.  "The 
nature  of  mass  belief  systems."  Pp. 
206-261  in  Ideology  and  Its 
Discontents,  edited  by  D.  Apter. 
New  York:  Free  Press;  Wuthnow, 
Robert.  1989.  Communities  of 
Discourse:  Ideology  and  Social 
Structure  in  the  Reformation,  the 
Enlightenment,  and  European 
T.  R.  Gurr.  Why  men  rebel. 
Princeton  University  Press, 


Peter  L.  Berger  and  Thomas 
Luckmann  in  1 966  book  The  Social 
Construction  of  Reality 


Conceptual  Model 
Type  Description  Value 

PeerRvw  Peer  Reviewed  3 

Theory 


PeerRvw  Peer  Reviewed  3 

Theory 


SWAG+  SWAG  with  2.5 

some  review 


SWAG+  SWAG  with  2.5 

some  review 

PeerRvw  Peer  Reviewed  3 

Theory 


SWAG  Scientific  Wild  2 

Assed  Guess 


PeerRvw  Peer  Reviewed  3 

Theory 


SWAG+  SWAG  with  2.5 

some  review 
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DisciplineName 
Theory  Name 

SocialEmotion 


Socialldentity 

SociallmpactTheory 

StructuralF  unctional 

StructurationTheory 

Symboliclnteraction 


Subject  Matter  Expert 

HartleyDIME/PMESII 


Hayes  &  Sands 


Theory  Description 

The  theory  that  emotion  underlies  human 
reasoning  and  all  social  processes.  These 
references  draw  upon  multiple  models. 


the  psychological  basis  of  intergroup 
discrimination.  It  is  composed  of  four  elements: 

Categorization:  We  often  put  others  (and 
ourselves)  into  categories. 

Social  Impact  Theory.  The  theory  predicts  that 
as  strength  and  immediacy  increase  within  a 
group  (up  to  about  4-5  people),  conformity  will 
also  increase.  The  more  important  a  group  is  and 
Structural-Functionalism  focuses  on  the  ways 
social  institutions  meet  social  needs.  Attempts  to 
explain  social  cohesion  and  stability  of  a  society 
through  the  concept  of  solidarity.  In  more  "primitive" 
The  notion  of  construction  and  reconstruction 
of  the  social  system  through  human  interaction 
based  on  rules  and  resources 

Symbolic  Interactionism  is  a  theoretical 
approach  to  understanding  the  relationship 
between  humans  and  society.  The  basic  notion  of 
symbolic  interactionism  is  that  human  action 


Theory  Citation 

Collins,  Randall.  1981.  "On  the 
microfoundations  of 
macrosociology."  American  Journal 
of  Sociology  86:984-1014.;  — . 
1990.  "Stratification,  emotional 
energy  and  the  transient  emotions." 
Pp.  27-57  in  Research  Agendas  in 
the  Sociology  of  Emotions,  edited 
by  T.  D.  Kemper.  Albany:  SUNY 
Press.;  — .  1993.  "Emotional 
energy  as  the  common 
denominator  of  rational  choice." 
Rationality  and  Society  5:203-230.; 
— .  2004.  Interaction  Ritual 
Chains.  Princeton,  NJ:  Princeton 
University  Press;  Lawler,  Edward  J. 
2003.  "Interaction,  emotion  and 
collective  identities."  in  Advances 
in  Identity  Theory  and  Research, 
edited  by  P.  J.  Burke.  New  York: 
Kluwer  Academic;  Sallach,  David  L. 
2008.  "Modeling  emotional 
dynamics:  Currency  versus  field." 
Rationality  and  Society  20:343- 
365;  Scheff,  Thomas.  1994. 
"Emotions  and  identity:  A  theory 
of  ethnic  nationalism."  in  Social 
Theory  and  the  Politics  of  Identity, 
edited  by  C.  Calhoun.  Cambridge, 

Henri  Tajfel  and  John  Turner 


Latane,  1981 


Durkheim,  Hoult  1969,  Auguste 
Comte,  Radcliffe-Brown,  Layton 


Herman  and  Reynolds  1994 


Conceptual  Model 
Type  Description  Value 

PeerRvw+  Well  reviewed  3.5 

theory 


PeerRvw  Peer  Reviewed  3 

Theory 


SWAG+  SWAG  with  2.5 

some  review 

PeerRvw+  Well  reviewed  3.5 

theory 

SWAG+  SWAG  with  2.5 

some  review 

PeerRvw  Peer  Reviewed  3 

Theory 


DIME/PMESII  connections  added  by  Hartley 


DIME/PMESII  connections  defined  in  book 


Hartley,  Dean  S.,  III.  Operations 
Other  Than  War  (OOTW)  Flexible 
Asymmetric  Simulation 
Technologies  (FAST)  Prototype 
Toolbox:  ISSM  v4.00  Analysts’ 

Hayes,  Bradd  C.  and  Jeffrey  I. 
Sands.  Doing  Windows:  Non- 
Traditional  Military  Responses  to 
Complex  Emergencies.  CCRP, 


SWAG  Scientific  Wild  2 

Assed  Guess 


SWAG+  SWAG  with  2.5 

some  review 
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5.2  DATA  SOURCES 


Data  Source  Name 
Data  Set  Name  Data  Description 


Data  Citation 


Data  Producer  Validity 
Type  Description  Value 


Anti-Defamation  League 

AntiDefamationTerr  Anti-Defamation  League  Terrorism  site  http://www.adl.org/terrorism/symbols/  SomeFlaws  A  few  flaws  and  2.5 

some  suspected 


ARDA 

ARDA  The  Association  of  Religion  Data  Archives 

(ARDA)  strives  to  democratize  access  to  the  best 
data  on  religion.  Founded  as  the  American 
Religion  Data  Archive  in  1 997  and  going  online 

Barro  &  Lee 

EducationAttainment  Provides  comparative  levels  of  educational 

attainment  across  118  countries  and  45  years, 
including  no  schooling,  primary  school, 
secondary  school,  and  higher  school.  The  dataset 


http://www.thearda.com/Archive/Bro  ModA&P  Moderate  3.5 

accuracy  & 


CID- World  Bank  Data  Surfer  SomeFlaws  A  few  flaws  and  2.5 

http://paradocs.pols.columbia.edu/dat  some  suspected 

avi  ne/MainFrameSet.jsp;  updates 

from  Center  for  International 

Development  at  Harvard  University 

http://www.cid.harvard.edu/ciddata/ci 

ddata.html 


CenterForNonproliferationStudies 


CNS 


Central  Banks 

Central 


Center  for  Nonproliferation  Studies  (CNS) 
Terrorism,  WMD  and  Emergency  Preparedness 
The  Center  for  Nonproliferation  Studies  (CNS) 
strives  to  combat  the  spread  of  weapons  of  mass 


Economic  and  Socio  Demographic  Data  at  the 
National  Level  URL  for  Central  Banks  of  most 


http://cns.miis.edu/research/wtc01/te  ModA&P  Moderate  3.5 

rrwmd.htm  accuracy  & 


http://www.bis.org/cbanks.htm 


SomeFlaws  A  few  flaws  and  2.5 
some  suspected 


CIDCM 


CIDCM 

University  of  Maryland's  Center  for 

International  Development  and  Conflict 

http://www.cidcm.umd.edu/ 

ModA&P 

Moderate 
accuracy  & 

3.5 

ConnectednessMem 

Connected  Memberships  in  Inter-Govemmental 
Organizations.  These  data  provide  an  indicator 
for  "connectedness,"  or  how  wellintegrated  the 
state  is  with  the  international  system  and/or  its 

Monty  Marshall,  CIDCM,  The 
University  of  Maryland 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

Coup  d'Etat  Events 

This  data  compiles  basic  descriptive  information 
on  all  coups  d'etat  occurring  in  countries  with 
populations  greater  than  500,000  during  the 
period  1960-2003.  A  coup  d'etat  is  defined  as  a 

Monty  G.  Marshall  and  Donna 
Ramsey  Marshall,  Center  for 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

DirectMilitarylnterv 

This  compiled  dataset  focuses  on  identifying 
what  countries  or  international  organizations 
made  the  decision  to  intervene  militarily  in  a 

Monty  Marshall,  CIDCM,  The 
University  of  Maryland,  1 999 

FlawsKno 

Known  flaws  in 
data  production 

2 

specific  country  in  a  given  year.  If  an  actor  was 
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Data  Source  Name 
Data  Set  Name  Data  Description 

GroupDisciriminati  The  dataset  contains  indicators  of  group 

discrimination  and  separatism  including  group 
number,  country  and  group  names,  political  and 
economic  discrimination  indices,  separatism 
MajorArmedConflic  Major  armed  conflicts  and  conflict  regions, 

1946-2003  contains  data  on  seven  categories  of 
armed  conflicts:  (IV)  international  violence, 
(IW)  international  war,  (IN)  international 

Minorities  at  Risk  The  Minorities  at  Risk  database  provides  a 

combination  of  qualitative  and  quantitative 
information  concerning  all  communal  groups 
which  meet  the  criteria  for  inclusion  as  a 


Polity  IV  coded  annual  information  on  regime  and 

authority  characteristics  for  all  independent 
states  (with  greater  than  500,000  total 
population)  in  the  global  state  system  and  covers 

Correlates  of  War 

CorrelatesOfWar  Conflict  Data:  Inter  and  Intra  State  Correlates 

of  War  Project:  1816  -2006  State  System 
membership,  militarized  disputes,  formal 
alliances,  national  material  capabilities.  Not 

CulturalDataSet  This  data  set  records  the  size  of  ethnic  groups, 

linguistic  populations,  and  religious  bodies  in  the 
Correlates  of  War  interstate  system  for  each 
decade  from  1820  through  1990.  In  addition,  it 

Internat&CivilWar  Updates  for  these  data  are  part  of  the  Correlates 

of  War  2  project.  The  updated  data  included  here 
were  derived  from  three  separate  datasets:  COW 
Inter-State  War  Data,  1816-1997  (v3.0),  COW 

NationalMaterialCa  National  Material  Capabilities  Data.  This  dataset 
codes  for  each  international  system  member  for 
each  year  between  1816  and  200 1  demographic, 
military  and  industrial  indicators.  These 

Council  on  Foreign  Relations 

CouncilForgnRelati  Council  on  Foreign  Relations 


Crisis  Group 

IntntlCrisisGroup  International  Crisis  Group  an  independent,  non¬ 

profit,  non-governmental  organization,  with 
over  1 10  staff  members  on  five  continents, 
working  through  field-based  analysis  and  high- 

DatabanksInternational.com 

Bank's  Cross  Nationa  Bank’s  Cross  National  Time  Series  Data  Archive 
1815-2006  Demographic,  Economic, 
Development,  Stability,  Regime  Duration  data. 

Earth  Trends 

Earth  Trends  A  compilation  of  a  variety  of  social.  Economic 

and  environmental  data 


Encyclopedia  Brittanica 

GeoAnalyzer  GeoAnalyzer  (Encyclopedia  Brittanica),  Country 

Snapshots  offer  statistics  on  demography,  vital 
statistics,  national  economy,  transportation  and 
communication,  education  and  health,  and 


Data  Producer  Validity 

Data  Citation  Type  Description  Value 

Ted  R.  Gurr,  CIDCM,  The  SomeFlaws  A  few  flaws  and  2.5 

University  of  Maryland,  as  updated  some  suspected 

by  the  Political  Instability  Task 

Monty  Marshall,  CIDCM,  The  SomeFlaws  A  few  flaws  and  2.5 

University  of  Maryland,  2002  some  suspected 

Original  data  from  Monty  G. 

Marshall,  "Major  Episodes  of 

Minorities  At  Risk  Data  Generation  SomeFlaws  A  few  flaws  and  2.5 

and  Management  Program  some  suspected 

(MARGene),  used  to  access  to  data 
in  the  MAR  dataset  developed  by 
Ted  Gurr  and  associates  (Gurr  et  al. 

1993;  Gurr  2000).  Data  available  at 
http://www.cidcm.umd.edu/inscr/mar/ 

http://www.cidcm.umd.edu/polity/  ModA&P  Moderate  3.5 

accuracy  & 


http://www.correlatesofwar.org/  SomeFlaws  A  few  flaws  and  2.5 

some  suspected 


Phil  Schafer,  University  of  Michigan  SomeFlaws  A  few  flaws  and  2.5 

some  suspected 


Correlates  of  War  2  homepage  SomeFlaws  A  few  flaws  and  2.5 

http://cow2.la.psu.edu/  some  suspected 


Correlates  of  War  2  homepage  SomeFlaws  A  few  flaws  and  2.5 

http://cow2.la.psu.edu/  some  suspected 


http://www.cff.org/publication/9237/  SomeFlaws  A  few  flaws  and  2.5 

some  suspected 


http://www.crisisgroup.org/home/ind  ModA&P  Moderate  3.5 

accuracy  & 


http://databanksintemational.com/  ModA&P  Moderate  3.5 

accuracy  & 


http://earthtrends.wri.org/searchable  SomeFlaws  A  few  flaws  and  2.5 

_db/index.php?step=countries&cID[  some  suspected 

]=6&theme=5&variab 


http://geoanalyzer.britannica.com/  FlawsSusp  Suspected  flaws  3 

in  data 
production 
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Data  Source  Name  Data  Producer  Validity 


Data  Set  Name 

Data  Description 

Data  Citation 

Type 

Description  Value 

Englebert 

StateLegitimacyAfri 

Measures  the  developmental  effects  of  the 
quality  and  extent  of  state  legitimacy  and  related 
variables  worldwide  and  within  Africa.  The 
dataset  covers  more  than  100  countries  from 

Website  of  Pierre  Englebert 
http://www.politics.pomona.edu/pen 
glebert/  From  Pierre  Englebert, 

2000.  "State  Legitimacy  and 
Development  in  Africa,"  Lynne 

Rienner  Publishers.  Available  at 
http://www.rienner.com/viewbook.cf 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

Europa  Publications 

AfricaSouthofSahar 

Africa  South  of  the  Sahara 

http://www- 

sul.  stanford.edu/depts/ ssrg/africa/ 
www.unesco.org/unesdi/index.php/en 
g/repertoire/tous  "Africa  South  of 
the  Sahara,  Europa  Publication 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

EuropaWorldYearbo 

Europa  World  Yearbook 

www.europaworld.com/  "The 

Europa  World  Yearbook,  London: 
England,  Europa  Publications 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

FarEast&Australasia 

The  Far  East  and  Australasia  1970-2000 

Reference  Library  Book  "The  Far 

East  and  Australasia,  London, 

Europa  Publications,  1970-2000" 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

MidEast&N  Africa 

The  Middle  East  and  North  Africa  1948-59, 

Reference  Library  Book  "The 

Middle  East  and  North  Africa, 

London,  Europa  Publications,  1974- 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

Fearon 

N  oncontiguousState 

This  dataset  includes  an  indicator  of  states 
defined  as  noncontiguous.  Specifically,  countries 
with  territory  holding  at  least  10,000  people  and 
separated  from  the  land  area  containing  the 

James  D.  Fearon  and  David  D. 

Laitin,  “Ethnicity,  Insurgency,  and 

Civil  War,”  paper  presented  at  the 

2001  Annual  Meetings  of  the 

American  Political  Science 

Association,  San  Francisco,  CA 
(August  2001).  Data  sent  via  e-mail 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

Freedom  House 

Freedom  House 

Freedom  House  1972-2007  Civil,  Political  and 
Economic  freedom  and  Press  freedom 

http://www.freedomhouse.org/templa 

te.cfm?page=15 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

PoliticalRightsIndex 

Measure  of  rights  to  participate  meaningfully  in 
the  political  process 

http://www.freedomhouse.org/upload 

s/special_report/29.pdf 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

Global  Security.org 

GlobalSecurity 

A  news  portal  on  security  issues 

http://www.globalsecurity.org/index. 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

Global  Terror  Alert 

GlobalT  error  Alert 

A  private  web  site  purporting  to  offer  terrorism 

http://www.globalterroralert.com/ 

FlawsKno 

Known  flaws  in 
data  production 

2 

Colder 

DemocraticElectoral 

This  dataset  covers  the  electoral  institutions  used 
in  all  of  the  democratic  legislative  and 
presidential  elections  in  more  than  150  countries 
between  1946  (or  independence)  and  2000.  The 

Matt  Golder,  2005  (forthcoming). 
"Democratic  Electoral  Systems 

Around  the  World,  1946-2000." 

Electoral  Studies  24,  103-121.  Data 
and  paper  available  at 
http://homepages.nyu.edu/~mrg2 1 7/ 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 
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Data  Source  Name 
Data  Set  Name  Data  Description 


Data  Citation 


Data  Producer  Validity 
Type  Description  Value 


Harff  &  Gurr 


EliteCharacteristics 

The  codings  on  this  set  of  variables  are  based  on 

Barbara  Harff,  U.S.  Naval  Academy, 

SomeFlaws 

A  few  flaws  and 

2.5 

the  class  affiliations  of  the  chief  executive  and, 

as  updated  by  the  Political 

some  suspected 

if  information  is  available  on  his/her  immediate 
associates.  Variables  coded  include  the  ethnic 

Instability  Task  Force 

Hartley 

BattleofBritain 

1 8  Battle  of  Britain  air  battles 

Predicting  Combat  Effects,  Dean  S. 

FlawsSusp 

Suspected  flaws 

3 

Hartley  III,  INFORMS/MAS,  2001, 

in  data 

http://topicsinor.pubs.informs.org/pc 

production 

BrasseyBattles 

108  historical  battles  from  Brassey 

Predicting  Combat  Effects,  Dean  S. 

SomeFlaws 

A  few  flaws  and 

2.5 

Hartley  III,  INFORMS/MAS,  2001, 
http://topicsinor.pubs.informs.org/pc 

some  suspected 

CAA  LWDB 

627  historical  battles  from  1 600  to  1 944  from 

Predicting  Combat  Effects,  Dean  S. 

ModA&P 

Moderate 

3.5 

the  Center  for  Army  Analyses  (CAA)  Land 

Hartley  III,  INFORMS/MAS,  2001, 
http://topicsinor.pubs.informs.org/pc 

accuracy  & 

CivilWarBattles 

49  Civil  War  battles 

Predicting  Combat  Effects,  Dean  S. 

SomeFlaws 

A  few  flaws  and 

2.5 

Hartley  III,  INFORMS/MAS,  2001, 
http://topicsinor.pubs.informs.org/pc 

some  suspected 

Helm83 

83  historical  battles  from  Helmbold 

Predicting  Combat  Effects,  Dean  S. 

FlawsSusp 

Suspected  flaws 

3 

Hartley  III,  INFORMS/MAS,  2001, 

in  data 

http://topicsinor.pubs.informs.org/pc 

production 

Helm92 

92  historical  battles  from  Helmbold 

Predicting  Combat  Effects,  Dean  S. 

FlawsSusp 

Suspected  flaws 

3 

Hartley  III,  INFORMS/MAS,  2001, 

in  data 

http://topicsinor.pubs.informs.org/pc 

production 

HelmCW 

19  Civil  War  battles  from  Helmbold  (with 

Predicting  Combat  Effects,  Dean  S. 

SomeFlaws 

A  few  flaws  and 

2.5 

duplications  from  different  authorities) 

Hartley  III,  INFORMS/MAS,  2001, 
http://topicsinor.pubs.informs.org/pc 

some  suspected 

Inchon 

19  combat  days  from  Inchon  Campaign  of 

Predicting  Combat  Effects,  Dean  S. 

FlawsSusp 

Suspected  flaws 

3 

Hartley  III,  INFORMS/MAS,  2001, 

in  data 

http://topicsinor.pubs.informs.org/pc 

production 

OsipovBattles 

38  historical  battles  from  Osipov 

Predicting  Combat  Effects,  Dean  S. 

FlawsSusp 

Suspected  flaws 

3 

Hartley  III,  INFORMS/MAS,  2001, 

in  data 

http://topicsinor.pubs.informs.org/pc 

production 

ShortBattles 

72  short  historical  battles 

Predicting  Combat  Effects,  Dean  S. 

SomeFlaws 

A  few  flaws  and 

2.5 

Hartley  III,  INFORMS/MAS,  2001, 
http://topicsinor.pubs.informs.org/pc 

some  suspected 

WWIBattles 

12  WWI  battles 

Predicting  Combat  Effects,  Dean  S. 

SomeFlaws 

A  few  flaws  and 

2.5 

Hartley  III,  INFORMS/MAS,  2001, 
http://topicsinor.pubs.informs.org/pc 

some  suspected 

Human  Relations  Area  Files 

HRAFArchaeology 

Database  of  prehistory  of  the  world.  The  eHRAF 

Hosted  at  Yale. 

GoodA&P 

Accuracy  & 

4 

Collection  of  Archaeology  currently  covers 

http://www.yale.edu/hraf/collections. 

precision  slightly 

over  41  major  archaeological  traditions  with 
more  than  50,000  pages.  Each  tradition  file  also 

less  than  rqmts 

HRAFWorldCulture 

Previously  HRAF  Ethnography. 

Hosted  at  Yale. 

GoodA&P 

Accuracy  & 

4 

http://www.yale.edu/hraf/collections.htm  It  is 

http://www.yale.edu/hraf/collections. 

precision  slightly 

important  to  note  that  cultures  in  the  microfiche 
and  electronic  versions  of  the  HRAF  Collection 

less  than  rqmts 

Institute  for  Counter  Terrorism 

InstForCounterTerr 

Institute  for  Counter  Terrorism 

http://www.ict.org.il/ 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

International  Historical  Statistics 

IntnlHistoricalStati 

“International  Historical  Statistics”  Mitchell 

Reference  Library  Books 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

International  Monetary  Fund 

DirectionsOfT  rade( 

The  complete  DOT  dataset  includes 

International  Monetary  Fund 

SomeFlaws 

A  few  flaws  and 

2.5 

approximately  55,000  country  and  area  time 

Publication  Services  (via  CD-ROM) 

some  suspected 

series  data  covering  the  distribution  of  exports 
and  imports  by  partner  country  for  about  1 80 
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Data  Source  Name  Data  Producer  Validity 


Data  Set  Name 

GovtFinanclStatistic 

Data  Description 

Economic  and  Socio  Demographic  Data  at  the 
National  Level 

Data  Citation 

www.imf.org/extemal/pubs/ft/gfs/ma 

nual/gfs.htm 

Type  Description  Value 

SomeFlaws  A  few  flaws  and  2.5 

some  suspected 

IntnlFinanclStatisti 

International  Financial  Statistics  (IFS)  is  the 
International  Monetary  Fund's  principal 
statistical  publication,  published  on  CD-ROM 
monthly  with  data  from  January  1948.  The  CD- 

http://www.imfstatistics.org/imf/logo 
n.aspx  International  Financial 

Statistics  and  International 

Monetary  Fund  Standby  Agreements 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

International  National  Statistics 

IntnlN  ationalStatist 

International  National  Statistics  1950-1993 

"International  National  Statistics, 

UK,  MacMillan  Reference,  LTD. 

1998  [Europe;  Africa,  Asia  & 

Oceana;  The  Americas,  1950-1993" 

ModA&P  Moderate 
accuracy  & 

3.5 

InternationalLabourOrganization 

LABORSTA 

From  the  ILO  LABORSTA  service,  these  data 
cover  strikes  and  lockouts  from  1969-2003, 
including  variables  for  event  counts  and  numbers 
of  workers  involved.  The  ILO  defines  a  strike  as 

ILO  LABORSTA  Internet 
http://laborsta.ilo.org/ 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

IntOrgForStandardization 

IS03166 

ISO  3166  is  the  international  standard  for 
country  codes.  ISO  3166  encompasses  three 
parts.  ISO  3166-1:  This  contains  the  codes  that 
most  users  know  as  the  ISO's  country  codes.  This 

International  Organization  for 
Standardization,  Maintenance 

Agency  for  ISO  3 1 66  Country  Codes 

The  short  country  names  from  ISO 
3166-1  and  the  alpha-2  codes  are 
made  available  by  ISO  at 
http://www.iso.ch/iso/en/prodsservice 

ExcA&P  Accuracy  & 

precision  meet 
or  exceed  rqmts 

5 

IntTelecomUnion  (ITU) 

TelecomlndicatorData 

The  World  Telecommunication  Indicators 

Database  contains  time  series  data  for  the  years 

1960,  1965,  1970  and  annually  from  1975- 
2001.  The  total  dataset  covers  approximately 

International  Telecommunication 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

IRIS,  UnivOfMD 

IntnlCountryRiskG 

This  dataset  was  assembled  by  IRIS  from  hard 
copies  of  the  International  Country  Risk  Guide,  a 
monthly  publication  of  Political  Risk  services. 

Each  variable's  value  for  a  given  country  and 

Institute  for  Reform  and  the 

Informal  Sector  (IRIS),  University 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

Jamestown  Foundation 

TheJamestovvnFoun 

The  Jamestown  Foundation's  mission  is  to 
inform  and  educate  policy  makers  and  the 
broader  policy  community  about  events  and 
trends  in  those  societies  which  are  strategically 

http  ://www.j  amestown.org/ 

ModA&P  Moderate 
accuracy  & 

3.5 

KEDS 

KEDS 

Kansas  Events  Data  System  KEDS  computer 
program  that  encodes  news  reports  to  generate 
data  about  the  behavior  of  dissidents  toward 
governments,  governments  toward  dissidents, 

Philip  A.  Schrodt,  Shannon  G.  Davis 
and  Judith  L.  Weddle.  “Political 

Science:  KEDS — A  Program  for  the 
Machine  Coding  of  Event  Data.” 

Social  Science  Computer  Review. 

ModA&P  Moderate 
accuracy  & 

3.5 

Levy 

StateCapacitySurve 

In  1999,  the  Political  Instability  Task  Force 
decided  to  experiment  with  a  new  way  to 
measure  state  capacity  that  relied  on  a  survey  of 
country  experts.  Under  the  direction  of  Marc 

Marc  Levy,  CIESIN,  Columbia 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

Ludwig 

King  of  the 

This  data  set  contains  information  on  the  rulers 
of  all  countries  during  the  20th  century, 
highlighting  facets  of  political  leadership  that 

Database  for  materials  gathered  for 
the  book  King  of  the  Mountain: 

The  Nature  of  Political  Leadership 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

explore  the  motivations  behind  the  desire  to 
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Data  Source  Name 
Data  Set  Name  Data  Description 

Mickolus  &  Flemming 

ITERATE  International  Terrorism:  Attributes  Of  Terrorist 

Events  (ITERATE)  Data  cover  1968-2003; 
variables  describe  attributes  of  terrorist  events, 
including  count  of  number  of  incidents  per  year, 


MIPT 

MIPTTerrorismKnow  MIPT  Terrorism  Knowledge  Base 


Nation  Master 

NationMaster  NationMaster,  statistics 


NATO 

NATO  Explains  NATO's  contribution  to  the  fight 

against  terrorism  through  military  operations  in 
Afghanistan,  the  Balkans  and  the  Mediterranean 
and  by  taking  steps  to  protect  its  populations 


Northeast  Intelligence  Network 

NortheastlntlNetwork  Northeast  Intelligence  Network 

NYU 

LogicOfPoliticalSur  The  Logic  of  Political  Survival  attempts  to 
address  why  leaders  who  produce  peace  and 
prosperity  are  turned  out  of  office  after  only  a 
short  time,  while  those  who  produce  corruption, 


OECD 

Internatnl  SOPEMI  contains  international  migration  data 

for  OECD  member  countries  and  some  non¬ 
member  economies.  The  publication's  Statistical 
Annex  includes  outflows  and  inflows  of  foreign 


PCS 

PCS 


Project  Civil  Strife  (PCS)  uses  computerized 
coding  technology  to  generate  disaggregated  data 
useful  for  testing  the  hypotheses  advanced  by 
the  new  generation  of  intrastate  conflict 


Pew  Research  Center 

ThePewResearchCe  The  Pew  Global  Attitudes  Project  is  a  series  of 
worldwide  public  opinion  surveys.  More  than 
90,000  interviews  in  50  countries  have  been 
conducted  as  part  of  the  project.  Includes 


Data  Producer  Validity 

Data  Citation  Type  Description  Value 


Edward  F.  Mickolus  and  Peter  A.  SomeFlaws  A  few  flaws  and  2.5 

Flemming  (via  e-mail)  some  suspected 


Developed  by  the  National  SomeFlaws  A  few  flaws  and  2.5 

Memorial  Institute  for  the  some  suspected 

Prevention  of  Terrorism  (MIPT), 
the  Terrorism  Knowledge  Base 
offers  in-depth  information  on 


http://www.nationmaster.com  SomeFlaws  A  few  flaws  and  2.5 

some  suspected 


http://www.nato. int/issues/terrorism/i  SomeFlaws  A  few  flaws  and  2.5 

ndex.html  some  suspected 


http://www.homelandsecurityus.com/  SomeFlaws  A  few  flaws  and  2.5 

terrorgroups.asp  some  suspected 


The  Logic  of  Political  Survival  Data  SomeFlaws  A  few  flaws  and  2.5 

Source  some  suspected 

http://www.n5m.edu/gsas/dept/p0litics 
/data/bdm2s2/Logic.htm  From  Bruce 
Bueno  de  Mesquita,  Alastair  Smith, 

Randolph  Siverson,  James  Morrow, 

2002.  "The  Logic  of  Political 
Survival."  The  book  is  now  available 
from  MIT  Press  at 
http://mitpress.mit.edu./catalog/item 
/default.asp?sid=2D79498B- 1 CF9- 


Organisation  for  Economic  Co-  SomeFlaws  A  few  flaws  and  2.5 

Operation  and  Development  some  suspected 


Stephen  M.  Shellman,  College  of  SomeFlaws  A  few  flaws  and  2.5 

William  &  Mary  some  suspected 


http  ://pewglobal  .org/ 


SomeFlaws  A  few  flaws  and  2.5 
some  suspected 
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Data  Source  Name 
Data  Set  Name  Data  Description 


PoliticallnstabilityTaskForce 


IslamicFamily  Lawln 


Islamism  Dataset 


ReligiousFractionali 


WorldEthnicityData 


Covering  approximately  45  countries,  this 
dataset  provides  coded  indicators  describing  type 
of  personal  code  administration,  family  law  as  a 
source  of  political  controversy,  and  type  of 

This  dataset  contains  several  indicators  of 
Islamism  and  factors  related  to  Islamism  for 
selected  countries,  including:  Theological 
foundations  of  Islamism,  Islamism  as  a  religious 
Provides  statistics  through  2000  on  percentage 
of  population  following  various  religions  or 
religious  practices.  Also  includes  religion 
Herfindahl  index  and  an  indicator  of  whether 
This  dataset  captures  the  largest  ethnic  group  in 
each  selected  country,  ethnic  groups  represented 
in  those  countries  as  a  percentage  of  total 
population,  and  also  an  ethnic  diversity  index. 


PRIO 


Armed  Conflict  International  Peace  Research  Institute,  Oslo 

(PRIO),  Department  of  Peace  and  Conflict 
Research,  Uppsala  University  Armed  conflict 
remains  a  serious  problem  in  the  post-Cold  War 


Regan 

ThirdPartylnterventi  These  are  recorded  data  on  civil  conflicts  since 
1945,  and  any  associated  interventions  into 
those  conflicts.  These  data  were  meant  to 
provide  a  concrete  view  of  the  relationship 


Relief  Web 

ReliefWebHealth  ReliefWeb  is  the  world’s  leading  on-line  gateway 

to  information  (documents  and  maps)  on 
humanitarian  emergencies  and  disasters.  An 
independent  vehicle  of  information,  designed 

Schmeidl  &  Jenkins 

DisplacementDatase  Estimates  of  the  numbers  of  internally  displaced 
people  based  on  various  sources  (including  the 
United  Nations  High  Commissioner  for  Refugees, 
the  U.S.  Committee  for  Refugees,  the  U.S. 

Refugee  Dataset  Annual  totals  of  international  refugees  by 

country  of  origin,  based  on  various  sources 
(including  the  United  Nations  High 
Commissioner  for  Refugees,  the  U.S.  Committee 


SITE  Institute 

SITE  Institute 


SMEs 

Hartley,  Dean 


Summaries  of  materials  distributed  to  SITE 
Institute  intel  subscribers.  Includes  terrorism 
library  and  links  to  terrorist  web  sites 


Holdsworth,  David  SOCOM  Contractor 


Data  Producer  Validity 

Data  Citation  Type  Description  Value 


Commissioned  for  the  Political 
Instability  Task  Force 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

Commissioned  for  the  Political 
Instability  Task  Force 

FlawsKno 

Known  flaws  in 
data  production 

2 

Commissioned  for  the  Political 
Instability  Task  Force 

FlawsKno 

Known  flaws  in 
data  production 

2 

Commissioned  for  the  Political 
Instability  Task  Force 

FlawsKno 

Known  flaws  in 
data  production 

2 

Mikael  Eriksson  and  Peter  SomeFlaws  A  few  flaws  and  2.5 

Wallensteen,  2004.  Armed  Conflict  some  suspected 

1989-2003,  Journal  of  Peace 
Research  41(5);  Nils  Petter 
Gleditsch,  Peter  Wallensteen, 

Mikael  Eriksson,  Margareta 
Sollenberg,  and  Havard  Strand,  2002. 

Armed  Conflict  1946-2001 :  A  New 
Dataset,  Journal  of  Peace  Research 
39(5):  615-637;  Mikael  Eriksson, 

Peter  Wallensteen,  and  Margareta 
Sollenberg,  2003.  Armed  Conflict 
1989-2002,  Journal  of  Peace 
Research  40(5):  593-607.  Data 


Patrick  M.  Regan.  "Third  Party  SomeFlaws  A  few  flaws  and  2.5 

Interventions  and  the  Duration  of  some  suspected 

Intrastate  Conflict",  Journal  of 
Conflict  Resolution,  February  2002 
(available  from 

http://bingweb.binghamton.edu/%7E 


http://www.reliefweb.int/rw/dbc.nsf/d  SomeFlaws  A  few  flaws  and  2.5 

oclOO?OpenForm  some  suspected 


Susanne  Schmeidl,  York  University, 
and  J.  Craig  Jenkins,  Ohio  State 


Susanne  Schmeidl,  York  University, 
and  J.  Craig  Jenkins,  Ohio  State 


http  ://siteinstitute  .org/  index  .html 

Hartley  Consulting 
Alion  Sciences 


SomeFlaws  A  few  flaws  and  2.5 
some  suspected 


FlawsKno  Known  flaws  in  2 

data  production 


FlawsSusp 

Suspected  flaws 
in  data 

3 

production 

ModA&P 

Moderate 
accuracy  & 

3.5 

ModA&P 

Moderate 
accuracy  & 

3.5 
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Data  Source  Name 
Data  Set  Name  Data  Description 

Southeast  Asia  Terrorism  Portal 


Data  Citation 


Data  Producer  Validity 
Type  Description  Value 


South  AsiaT  errorism 

South  Asia  Terrorism  Portal  A  link  to  reports, 
studies,  etc  focusing  on  South  Asia  terrorism 

http://www.satp.org/default.asp 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

StanfordUniversity 

EthnicStructureCultu 

This  dataset  presents  a  list  of  some  820  ethnic 
groups  in  160  countries  that  made  up  at  least  1 
percent  of  country  population  in  the  early 

1 990s.  It  includes  a  measure  of  ethnic 

James  D.  Fearon,  "Ethnic  and 

Cultural  Diversity  by  Country," 

Journal  of  Economic  Growth  8,  2 
(June  2003):  195-222  Data  available 
at 

http://www.stanford.edu/group/ethnic 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

LeadershipDuration 

This  database  tracks  leadership  duration  across  a 
wide  range  of  countries  and  time.  Earliest  data 
extend  from  18th  century  through  1987.  Data 
from  1955-1987  were  originally  retained  for 

Henry  S.  Bienen  and  Nicolas  van  de 
Walle,  Of  Time  and  Power: 

Leadership  Duration  in  the  Modem 
World,  The  Center  of  International 
Studies,  Princeton  University;  as 
updated  by  the  Political  Instability 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

START 

GlobalT  errorismDat 

National  Consortium  for  the  Study  of  Terrorism 
and  Responses  to  Terrorism  (START).  The 

Global  Terrorism  Database  (GTD)  is  an  open- 
source  database  including  information  on 

http://www.start.umd.edu:80/data/gtd 

ModA&P 

Moderate 
accuracy  & 

3.5 

Taylor 

Political&Sociallndi 

This  new  edition  of  the  World  Handbook  is 
composed  of  four  files:  aggregate  data,  daily 
political  events,  annual  political  events,  and 
quarterly  political  events.  The  aggregate  data  file 

Charles  Lewis  Taylor,  Virginia 
Polytechnic  Institute  and  State 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

The  DataWeb 

TheDataWeb  TheDataWeb  is  network  of  online  data  libraries 

that  the  DataFerrett  application  access  the  data 
through.  Data  topics  include,  census  data, 
economic  data,  health  data,  income  and 

Transparency  International 

http://www.thedataweb.org/index.ht 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

TI  Corruption  Index 

Transparency  International  Corruption 

Perception  Index  1995  -  2007  Transparency 
across  revenue,  national  accounts,  perceived 

http://www.transparency.org/policy_ 

research/surveys_indices/cpi 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP 

MILC 

Managing  Intrastate  Low-intensity  Conflict 
(MILC)  v.  1.0  1993-2004  Third  party  actions  in 

http://www.pcr.uu.se/research/UCDP/ 

data_and_publications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP  Battle-Deaths 

UCDP  Battle-Deaths  Dataset  v.4.1,  2002-2005 
Conflict  level  and  conflict  year  data  on  number 
of  deaths  in  internal  conflicts. 

http://www.pcr.uu.se/research/UCDP/ 

data_and_publications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP  Dyadic 

UCDP  Armed  Conflict  Dyadic  v.4-2006,  1989  - 
2005  Dyadic  version  of  the  UCDP/PRIO  armed 

http://www.pcr.uu.se/research/UCDP/ 

dataandpublications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP  Non-State 

UCDP  Non-State  Conflict  Dataset  v.1.1,  2002  - 
2005  Data  set  on  internal  conflicts  where 
neither  warring  party  is  the  government 

http://www.pcr.uu.se/research/UCDP/ 

data_and_publications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP  One-Sided 

UCDP  One-Sided  Violence  Dataset  v.1.2  1989  - 
2005  Intentional  attacks  on  civilians  by  the 
government  or  third  party. 

http://www.pcr.uu.se/research/UCDP/ 

data_and_publications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP  Peace 

UCDP  Peace  Agreement  Dataset  v.  1.0,  1989- 
2005  Data  set  that  includes  peace  agreements 
signed  by  at  least  two  warring  parties. 

http://www.pcr.uu.se/research/UCDP/ 

data_and_publications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP  Termination 

UCDP  Conflict  Termination  dataset  v.2.0,  1946 
-  2006  Conflict  level  and  conflict  year  data. 

http://www.pcr.uu.se/research/UCDP/ 

data_and_publications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 

UCDP/PRIO 

UCDP/PRIO  Armed  Conflict  v.4-  2007,  1946  - 
2006  Conflict-year  data  within  countries  where 
at  least  one  party  is  the  government. 

http://www.pcr.uu.se/research/UCDP/ 

dataandpublications/datasets.htm 

ModA&P 

Moderate 
accuracy  & 

3.5 
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Data  Source  Name 
Data  Set  Name  Data  Description 


Data  Citation 


Data  Producer  Validity 
Type  Description  Value 


UN 

AfricanStatYearboo  "African  Statistical  Yearbook  "[ALSO  African 
National  Statistics,  ASN=ASY]" 

AQUASTAT  The  Food  and  Agriculture  Organization  of  the 

United  Nations  leads  international  efforts  to 
defeat  hunger.  FAO  is  also  a  source  of 
knowledge  and  information.  AQUASTAT  is 

Asia&FarEast68-69  Statistical  Yearbook  for  Asia  and  the  Far 


Asia&FarEast74-84,86  "Statistical  Yearbook  for  Asia  and  the  Pacific, 
United  Nations,  Bangkok,  Thailand,  Economic 
and  Social  Commission  for  Asia  and  the  Pacific, 
1974-84,  1986-90,  1992-1999" 

FAOSTAT  FAOSTAT  is  an  on-line  and  multilingual 

database  currently  containing  over  3  million 
time-series  records  covering  international 

IncomelnequalityDa  The  World  Income  Inequality  Database  collects 
and  stores  information  on  income  inequality  for 
developed,  developing,  and  transition  countries. 
Data  includes  the  GINI  Coefficient  in  percentage 

LatinAmerica75-84  "Statistical  Yearbook  for  Latin  America, 

Santiago,  Chile,  United  Nations,  Economic 
Commission  for  Latin  America,  1975-84" 

Latin America86-99  "Statistical  Yearbook  for  Latin  America  and  The 

Caribbean,  Santiago,  Chile,  Economic 
Commission  for  Latin  America  and  the 

LatinAmericaCEPA  “The  Statistical  Abstract  of  Latin  America 


Trade&Development  This  handbook  is  intended  to  provide  a 

comprehensive  collection  of  statistical  data 
relevant  to  the  analysis  of  world  trade, 
investment  and  development.  It  was  created  by 
UN  Human  Human  Development  Reports  2005-2008  Basic 

needs  and  quality  of  life  data  (UNDP). 

UN  Stat  Databases  UN  Statistical  Data  bases,  Yearbooks  &  Reports 


UNEnergyStatistics  The  database  contains  comprehensive  energy 

statistics  on  production,  trade,  and  consumption 
(end-use)  for  primary  and  secondary 
conventional,  non-conventional,  new,  and 

UNPeriodical  UN  periodical  report 

WorldData  access  to  many  UN  databases 

WorldPopulationPro  World  Population  Prospects:  The  2002  Revision 
contains  demographic  estimates  for  1950-2000 
and  four  variants  of  projections  for  2000-2050 
for  countries,  regions  and  major  areas  of  the 
World  Urbanization  The  World  Population  Prospects:  2003  Revision 

(urban  and  rural  areas  1950-2030,  urban 
agglomerations  1950-2015)  includes  data  for 
urban  populations  and  urban  agglomerations  with 


https://unp.un.org/details.aspx?pid=l  FlawsSusp  Suspected  flaws  3 

in  data 
production 

http://www.fao.Org/waicent/faoinfo/a  SomeFlaws  A  few  flaws  and  2.5 

gricult/agl/aglw/aquastat/dbase/index.s  some  suspected 


Reference  Library  Book  "Statistical  FlawsSusp  Suspected  flaws  3 

Yearbook  for  Asia  and  the  Far  East,  in  data 

Bangkok,  Thailand,  1968-1969,  production 

United  Nations" 

Reference  Library  Book  FlawsSusp  Suspected  flaws  3 

in  data 
production 

http://faostat.fao.org/  FlawsSusp  Suspected  flaws  3 

in  data 
production 

United  Nations  Development  SomeFlaws  A  few  flaws  and  2.5 

Programme,  Social  Development  &  some  suspected 

Poverty  Elimination  Division 

Reference  Library  Book  FlawsSusp  Suspected  flaws  3 

in  data 
production 

Reference  Library  Book  FlawsSusp  Suspected  flaws  3 

in  data 
production 

Reference  Library  Book  FlawsSusp  Suspected  flaws  3 

in  data 
production 

United  Nations  Conference  on  SomeFlaws  A  few  flaws  and  2.5 

Trade  and  Development  Handbook  some  suspected 

of  International  Trade  and 

http://hdr.undp.org/en/statistics/  FlawsSusp  Suspected  flaws  3 

in  data 
production 

http://unstats.un.0rg/unsd/databases.h  FlawsSusp  Suspected  flaws  3 

in  data 
production 

Hermann  Habermann,  Director,  FlawsKno  Known  flaws  in  2 

United  Nations  Statistical  Division,  data  production 


FlawsKno  Known  flaws  in  2 

data  production 

http://data.un.org:80/  SomeFlaws  A  few  flaws  and  2.5 

some  suspected 

Population  Division  of  the  SomeFlaws  A  few  flaws  and  2.5 

Department  of  Economic  and  some  suspected 

Social  Affairs  of  the  United  Nations 

Population  Division  of  the  SomeFlaws  A  few  flaws  and  2.5 

Department  of  Economic  and  some  suspected 

Social  Affairs  of  the  United  Nations 
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Data  Source  Name 
Data  Set  Name 

UN  WHO 

AIDSCasesDatabase 

Report  on  HIV/ AIDS 


WorldHealthOrgani 


UNC  Asheville 

PolitSciT  errorScale 


UNESCO 

UNESCOStatistics 

UnivOfPennsylvania 

Democracy&Develop 


PennWorldTables 

UOB  DAT 

NGOStructures 


OrderofBattleCFDB 


OrderofBattleCFE 


OrderofBattleFFDB 


OrderofBattleJCOFA 


Data  Description 


Data  Citation 


Data  Producer  Validity 
Type  Description  Value 


AIDS  data  reported  to  WHO  by  country/area. 
Although  coverage  may  vary  by  country,  the 
data  include  the  number  and  rate  of  reported 
AIDS  cases  from  1979  though  2002,  by  year  of 
From  the  July  2004  UNAIDS  Report  on  the 
Global  HIV/AIDS  Epidemic,  these  data  show  the 
HIV/AIDS  rate  among  adults  aged  15-49  at  the 


The  WHO  Statistical  Information  System  is  the 
guide  to  health  and  health-related 
epidemiological  and  statistical  information 
available  from  the  World  Health  Organization. 


World  Health  Organization, 
Department  of  HIV/AIDS  (as 
provided  by  Abiola  Johnson  on  CD- 

UNAIDS  2004  Report  on  the  Global 
AIDS  Epidemic  2004  The  data  are 
from  the  section  "Table  of  country- 
specific  HIV/AIDS  estimates  and 
data,  end  2003."  Available  at 
http://www.unaids.org/bangkok2004/ 
http://www3  .who.int/whosis/menu.cfm 


SomeFlaws  A  few  flaws  and 
some  suspected 


SomeFlaws  A  few  flaws  and 
some  suspected 


SomeFlaws  A  few  flaws  and 
some  suspected 


2.5 


2.5 


2.5 


This  is  a  graded  scale  for  measuring  human  rights 
violation  and  was  adapted  from  work  originally 
published  by  Raymond  Gastil  of  Freedom  House 
in  1979.  The  raw  information  comes  from  the 


Political  Science  Department,  SomeFlaws  A  few  flaws  and  2.5 

University  of  North  Carolina  some  suspected 

Asheville,  Mark  Gibney  Data 
available  at 

http://www.unca.edu/politicalscience/ 


Global  and  internationally  comparable  statistics 
on  education,  science,  technology,  culture  and 


http://www.uis.unesco.org/ev_en.php 
?ID=2867_20 1  &ID2=DO_TOPIC 


FlawsSusp  Suspected  flaws  3 

in  data 
production 


Democracy  and  Development:  Political 
Institutions  and  Material  Well-Being  in  the 
World.  This  dataset  covers  more  than  120 
countries  observed  between  1950  or  the  year  of 


Penn  World  Tables  Provides  purchasing  power 
parity  and  national  income  accounts  converted 
to  international  prices  for  188  countries  for 


Adam  Przeworski,  Michael  Alvarez, 
Jose  Antonio  Cheibub  and  Fernando 
Limongi.  Democracy  and 
Development:  Political  Institutions 
and  Material  Well-  Being  in  the 
World,  1950-1990.  Cambridge: 
www.imf.org/extemal/pubs/ft/gfs/ma 
nual/gfs.htm;  Data  available  at 
http://pwt.econ.upenn.edu/. 


SomeFlaws  A  few  flaws  and  2.5 
some  suspected 


ModA&P  Moderate  3.5 

accuracy  & 


Unclassified  Non-Govemmental  Organizations 


Classified  US  Units  from  OSD  PA&E’s 
Conventional  Forces  Data  Base  (CFDB) 


Unclassified  Conventional  Forces  Europe  (CFE) 


Classified  units  from  the  Future  Forces  Data  Base 


Classified  foreign  units  from  JCOFA 


Unit  Order  of  Battle  Data  Access 
Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 
Beauregard  St  Suite  500,  Alexandria 

Unit  Order  of  Battle  Data  Access 
Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 
Beauregard  St  Suite  500,  Alexandria 

Unit  Order  of  Battle  Data  Access 
Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 
Beauregard  St  Suite  500,  Alexandria 

Unit  Order  of  Battle  Data  Access 
Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 
Beauregard  St  Suite  500,  Alexandria 

Unit  Order  of  Battle  Data  Access 
Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 
Beauregard  St  Suite  500,  Alexandria 


SomeFlaws  A  few  flaws  and  2.5 
some  suspected 


ModA&P  Moderate  3.5 

accuracy  & 


FlawsSusp  Suspected  flaws  3 

in  data 
production 


FlawsSusp  Suspected  flaws  3 

in  data 
production 


FlawsSusp  Suspected  flaws  3 

in  data 
production 
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Data  Source  Name  Data  Producer  Validity 


Data  Set  Name 

Data  Description 

Data  Citation 

Type 

Description  Value 

OrderofBattleMIDB 

Classified  D1A  Modernized  Integrated  Data  Base 

Unit  Order  of  Battle  Data  Access 

Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 

Beauregard  St  Suite  500,  Alexandria 

ModA&P 

Moderate 
accuracy  & 

3.5 

OrderofBattleOpFor 

Unclassified  foreign  units  Opposing  Forces 

Unit  Order  of  Battle  Data  Access 

Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 

Beauregard  St  Suite  500,  Alexandria 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

OrderofBattleTUCHA 

Unclassified  US  forces  from  the  Global 

Command  and  Control  System  (GCCS)  Type 

Unit  Characteristics  file  (TUCHA)  and  the  Unit 

Unit  Order  of  Battle  Data  Access 

Tool  (UOB  DAT),  Computing 
Technologies  Inc.,  Christian  Farrell, 
cfarrell@dmso.mil,  1901  N 

Beauregard  St  Suite  500,  Alexandria 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

US  AID 

USAIDPeriodical 

US  AID  periodical  report 

FlawsKno 

Known  flaws  in 
data  production 

2 

US  Army 

FM  100.23-1 

Multiservice  Procedures  for  Humanitarian 

Assistance  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  100-20 

Military  Operations  in  Low  Intensity  Conflict 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  100-23 

Peace  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-0 

Army  Operations  manual 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-05.401 

Civil  Affairs  Tactics  Techniqus  and  Procedures 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-07 

Stability  Operations  and  Support  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-07.31 

Peace  Operations:  Multi-Service  Tactics, 

Techniques,  And  Procedures  For  Conducting 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-100.21 

Contractors  on  the  Battlefield 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-13 

Information  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-19.40 

Military  Police  Intemment/Resettlement 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-24 

Counterinsurgency 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  3-61.1 

Public  Affairs  Tactics,  Techniques  and  Procedures 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  4-0 

Combat  Service  Support  -  logistics,  supply, 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  5-100 

Engineer  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  90-29 

Noncombatant  Evacuation  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

FM  90-8 

Counterguerrilla  Operations 

ModA&P 

Moderate 
accuracy  & 

3.5 

US  Army  CAA 

ArdennesCampaign 

Center  for  Army  Analyses  (CAA)  data  on 

Ardennes  Campaign  1 944-45 

Ardennes  Campaign  Simulation 
(ARCAS),  US  Army  CAA,  1995, 

ModA&P 

Moderate 
accuracy  & 

3.5 

90 


Data  Source  Name  Data  Producer  Validity 


Data  Set  Name 

Data  Description 

Data  Citation 

Type 

Description  Value 

US  Census  Bureau 

HIV/AIDSSurveillan 

HIV/AIDS  Surveillance  data  base  The  HIV/AIDS 
Surveillance  Data  Base  was  developed  and  is 
maintained  by  the  Health  Studies  Branch, 

International  Programs  Center  (IPC), 

http://www.census.gov/ipc/www/hivai 
dsd.html,  U.S.  Bureau  of  the  Census, 
International  Programs  Center, 

Health  Studies  Branch  (via  CD- 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

InfantMortalityRate 

These  data  describe  the  infant  mortality  rate 
(IMR)  as  the  number  of  deaths  of  infants  under 
one  year  of  age  per  1 ,000  live  births  in  a  given 
year.  The  data  can  vary  for  many  reasons.  In  the 

U.S.  Bureau  of  Census,  International 
Programs  Center,  Population 

Division  (via  e-mail) 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

InternationalDataBas 

The  International  Data  Base  provides 
demographic  and  socio-economic  statistics  for 

227  countries  and  areas  of  the  world.  Major 
types  of  data  available  in  the  IDB  include 

U.S.  Bureau  of  Census,  International 
Programs  Center,  Population 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

PopulationPyramids 

US  Government  Census  Bureau:  Poplation 

http://www.census.gov/ipc/www/idbp 

ModA&P 

Moderate 
accuracy  & 

3.5 

USCensusBureau 

US  Census  Bureau  Data  Center,  Access  to  US 

Census  data  and  tools  supporting  research  on 

http  ://www.  census  .gov/main/www/acc 
ess.html 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

US  Commerce  Dept 

TradeStats 

US  Dept  of  Commerce  annual  and  quarterly  trade 

http://tse.export.gov/ITAHome.asp 
x?UniqueURL=gbgwon5  5  avf4xdmnl 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

sojew45-2006-2-3-l  0-48-35 


US  Dept  HomelandSecurity 


ImmigrationStatisti 

The  Yearbook  provides  immigration  data 
through  2002.  Major  areas  include:  immigrants 
admitted  for  legal  permanent  residence;  refugees 
approved  and  admitted;  nonimmigrant  arrivals; 

U.S.  Bureau  of  Citizenship  and 
Immigration  Services  (formerly 
Immigration  and  Naturalization 

Service  Available  at 
http://uscis.gov/graphics/shared/abou 

ModA&P 

Moderate 
accuracy  & 

3.5 

US  DoD 

Joint  Pub  5-0 

Joint  Operational  Planning 

ModA&P 

Moderate 
accuracy  & 

3.5 

US  DOE 

EnergyProdConsPric 

This  U.S.  Energy  Information  Administration 
data  contains  information  on  energy  production, 
consumption,  and  prices,  with  coverage  for 
most  variables  from  1980  to  2002  or  2003. 

U.S.  Energy  Information 

Administration,  Main  Products  of 
the  Office  of  Energy  Markets  and 

End  Use 

FlawsKno 

Known  flaws  in 
data  production 

2 

US  Government 

CIA  WFB 

CIA  World  Fact  Book,  General  reference  on 
political,  economic,  social,  cultural  data  by 

https://www.cia.gov/cia/publications/ 

factbook/index.html 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

JTCG/ME 

Munitions  effectiveness 

VGoodA&P 

Accuracy  & 
precision 
thought  to  meet 

4.5 

OPIC 

Overseas  Private  Investment  Corporation 
(OPIC)  OPIC  helps  U.S.  businesses  invest 
overseas,  fosters  economic  development  in  new 
and  emerging  markets,  complements  the  private 

http://www.opic.gov/ 

ModA&P 

Moderate 
accuracy  & 

3.5 

T  erroristGrpProfiles 

Index  of  Groups  published  in  the  Dudley  Knox 
Library,  Naval  Post  Graduate  School.  Material 
taken  from  Country  Reports  on  Terrorism  and 

Patterns  of  Global  Terrorism,  US  Dept  of  State 

http://library.nps.navy.mil/home/tgp 
/tgpndx.htm#200 1 

SomeFlaws 

A  few  flaws  and 
some  suspected 

2.5 

US  Institute  for  Peace 

USInstituteForPeace 

An  independent,  nonpartisan,  national 
institution  established  and  funded  by  Congress. 

Its  goals  are  to  help  prevent  and  resolve  violent 

http://www.usip.org/ 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

international  conflicts,  promote  post-conflict 
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Data  Source  Name  Data  Producer  Validity 


Data  Set  Name 

Data  Description 

Data  Citation 

Type  Description 

Value 

US  OFDA 

DisasterHistoryDatab 

The  disaster  history  database  is  maintained  by 
the  Office  of  U.S.  Foreign  Disaster  assistance.  It 
includes  information  on  various  natural  and 
humaninduced  disasters  including  drought, 

U.S.  Agency  for  International 
Development,  Office  of  U.S. 

FlawsKno  Known  flaws  in 
data  production 

2 

US  State  Dept 

CoordReconStabiliz 

Office  of  the  Coordinator  for  Reconstruction 
and  Stabilization  This  office  will  lead, 
coordinate,  and  institutionalize  U.S.  Government 
civilian  capacity  to  prevent  or  prepare  for  post- 

http://www.  state.gov/s/crs/rls/43 327 . 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

USStateDept 

Country  Background  Notes;  Facts  about  the  land, 
people,  history,  government,  political 
conditions,  economy,  and  foreign  relations  of 
independent  states,  some  dependencies,  and  areas 

http://www.state.gOv/r/pa/ei/bgn/ 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

US  Treasury 

USTreasury 

Terrorism  and  Financial  intelligence 

http://www.treasury.gov/offices/enfo 

FlawsSusp  Suspected  flaws 
in  data 
production 

3 

USMC 

Small  Wars  Manual 

how  to  operate  in  small  wars 

USMC  1940 

ModA&P  Moderate 
accuracy  & 

3.5 

Vanhanen 

Polyarchy  Dataset 

The  Polyarchy  dataset  codes  political 
competitiveness  and  participation  in  1 87 
countries  over  the  period  1810  to  1998.  This 
dataset  is  the  result  of  a  collaborative  project 

Tatu  Vanhanen,  University  of 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

Wharton 

PoliticalConstraint 

This  dataset  was  designed  to  derive  an  objective 
internationally  comparable  measure  of  the 
feasibility  of  policy  change  in  a  given  country 
for  a  given  year.  This  main  measure,  labeled  the 

Henisz,  Witold  Jerzy  2000.  "The 
Institutional  Environment  for 

Economic  Growth."  Economics  and 
Politics,  12(1):  1-31.  Data  available 
at  http://www- 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

World  Bank 

EffectDevelopAssist 

This  database  contains  the  conventional  and  the 
adjusted  measures  of  official  development 
assistance  to  a  set  of  133  countries  between 

1975  and  1995.  The  principal  component  of  the 

The  World  Bank  Group  Economic 
Growth  Research 

http://www.worldbank.Org/research/g 

rowth/ddaid.htm 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

GlobalDevelopFinan 

Global  Development  Finance  2004  is  the  World 

Bank's  review  of  the  external  debt  and  finance 
flows  of  developing  countries.  It  examines  recent 
developments  in  flows  from  international 

The  World  Bank 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

GovernanceMattersl 

This  dataset  contains  updated  composite 
governance  research  indicators  for  199 
countries,  covering  six  dimensions  of 
governance:  Voice  and  accountability,  political 

Group  and  World  Bank  Institute 
http  ://www.  worldbank.  org/wbi/gover 
nan  ce/govdata2002/index.html 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

Mountains 

The  World  Mountains  dataset  measures  the 
percentage  of  mountain  area  in  161  countries. 
Mountain  areas  were  identified  using  criteria 
including  elevation,  relative  relief,  and  an 

A.J.  Gerrard  and  Paul  Collier  for  The 
World  Bank  Conflict  Group,  with 

James  Fearon  and  David  Laitin 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

Politicallnstitution 

A  cross-country  database  of  political  institutions, 
this  data  covers  1 77  countries  over  26  years 
(1975-2000).  The  variables  provide  details  about 
elections,  electoral  rules,  type  of  political 

The  World  Bank  Group  Research 
http://www.worldbank.org/research/bi 
os/  pkeefer.htm 

SomeFlaws  A  few  flaws  and 
some  suspected 

2.5 

Urban  Governance 

The  Urban  Governance  Initiative  Report  Cards 

World  Bank  -  perception  of  institutional 
reliability,  corruption,  urban  &  rural 

http://web.w0rldbank.0rg/WBSITE/E 
XTERNAL/WBI/EXTWBIGOVAN 
TCOR/0„contentMDK:2074647  l~p 
agePK:64 1 68445~piPK:64 1 68309- 

FlawsSusp  Suspected  flaws 
in  data 
production 

3 

WorldBankGroupW 

population  with  access  to  potable  water 

http://devdata.worldbank.org/data- 

FlawsSusp  Suspected  flaws 
in  data 
production 

3 
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Data  Source  Name 
Data  Set  Name 

World  BankHistoric 

Data  Description 

Historical  infant  mortality  rate  data  for 
countries  no  longer  in  existence  (CZE,  DRV, 

RVN,  ETH,  GDR,  GFR,  USS,  YAR,  YPR,  YUG, 
PKS).  Data  was  compiled  using  the  World  Bank 

Data  Citation 

The  World  Bank  Development  Data 

Data  Producer  Validity 
Type  Description  Value 

SomeFlaws  A  few  flaws  and  2.5 

some  suspected 

WorldDevIndicators 

World  Development  Indicators  (WDI)  provides 
an  expanded  view  of  the  world  economy  for 
more  than  200  countries  with  populations  of 
over  one  million.  It  contains  data  on  the  people 

www.worldbank.org/data/wdi2004/ 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

World  Economic  Forum 

WorldEconomicForum 

Macroeconomic  environment  index 

http :  //www.  weforum.  org/en/  initiative 
s/gcp/GlobalCompetitivences 

ModA&P 

Moderate 
accuracy  & 

3.5 

World  Freedom  Atlas 

World  Freedom 

World  Freedom  Atlas  1990-2006  The  geo-world 
atlas  is  a  geo- visualization  tool  for  world 

http://freedom.indiemaps.com/ 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 

World  Values  Survey 

World  Values 

World  Values  Survey,  1981  -  2004  Surveys 
conducted  in  waves,  individual  perceptions  of 
regime  satisfaction,  economic  satisfaction,  well 

http://www.worldvaluessurvey.org/ 

FlawsSusp 

Suspected  flaws 
in  data 
production 

3 
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